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preface 


In 1957, Division 5 of the American Psychological Association estab- 
lished an ad hoc committee consisting of Ledyard Tucker, Harry Harmon, 
and myself to foster the use of computers in psychology. As part of the 
work of this committee, I prepared an expository report intended as an 
introduction to computing in psychology. It became clear, however, 
that something more than a short report was needed; so I resolved to 
write a book that would acquaint behavioral and social scientists with 
the use and operation of computers. Here, after many delays, is the 
product of that resolution. 

The book combines an introduction to computer programming with 
an account of the applications of computers to research problems. Either 
part can be read independently, but I feel strongly that the applications 
cannot be appreciated fully without an understanding of computer pro- 
gramming. Computers are novel, but they are relatively simple, and 
anyone with a good background in high-school mathematics can under- 
stand them. The complications can be left to the machine designers, 
and the sophisticated mathematics can be left to those with difficult 
quantitative problems. Computers are good for many more things than 
mathematical calculations and can be put through their paces by non- 
mathematicians. A scientist who understands how to use computers and 
knows how others have used them can rely on his own imagination to 
invent new applications in his own field of research. 

Part I deals with programming—the art of instructing a digital com- 
puter how to solve a problem. The basic elements of programming are 
presented in logical order, Index registers, subroutines, and symbolic 
coding are introduced early in the exposition, and a chapter is devoted 
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to programming languages. Many details have been omitted or post- 
poned. With one exception, the computer word is the smallest unit of 
computation in Part I. No distinction is made between binary and 
decimal computers. All digit-by-digit considerations are postponed, so 
that the student can get on board smoothly and swiftly. 

Part II takes up digital codes and considers the detailed structure of 
computer words. The distinction is made between binary and decimal 
computers, and the discussion is largely concerned with binary machines. 
Binary arithmetic is described, and fixed and floating-point numbers are 
defined. Methods are described for programming the computer to deal 
with arbitrarily encoded information. The input and output instruc- 
tions are considered in this part. 

The major uses of computers in psychology are discussed in Part III. 
Special problems that arise in numerical applications are treated, but 
nonnumeric applications are emphasized, since these are the novel uses 
that expand the power of computers. Stimulus generation, computer 
models of behavior, and man-computer interrelations are treated. The 
applications are not all-inclusive, but they are representative. I have 
discussed at length only those areas in which I have had experience or 
firsthand knowledge. Many of the examples have been drawn from my 
own research. Whenever a selection of material was necessary, I always 
chose the work with which I was more familiar. References are given to 
pertinent literature, much of which is located in obscure and unfamiliar 
places. 

Part IV is for the curious. It discusses the logical foundations of com- 
puters and gives an elementary explanation of J’uring machines, to 
provide a logical model of the computing process. 

The text has been supplemented by many illustrative examples, and 
problems have been listed at the end of most chapters to provide practice 
materials. The casual reader may safely omit the examples and problems, 
but the serious student should be sure he understands the examples and 
should try his hand at some of the problems. A few of the problems 
present some special issues not mentioned in the text. Others, especially 
in Part III, are projects suitable for bachelor’s and master’s theses. 

The reader will need no mathematics beyond high-school algebra and 
trigonometry, except for some illustrative examples that are specially 
marked. However, a knowledge of elementary statistics will be helpful. 
Some acquaintance with psychological research will also be helpful, since 
most of the sample programs and illustrative examples are drawn from 
the field of psychology. 

Some of the examples and problems are phrased at the level of flow 
diagrams, and no specific computer need be assumed, But some of the 
discussion must be concrete, and for this purpose a hypothetical com- 
puter has been invented. This computer is very much like the computers 
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with which I am most familiar: the IBM 704/709/7090, and the Lincoln 
Laboratory MTC and TX-2 computers. The fictitious computer is a 
single-address binary machine with a 36-bit word length; its instructions 
are defined in the Appendix. Because some details have been omitted, 
most of the instructions could apply equally to a decimal computer. 
Only the logical and in-out instructions, and one control instruction, 
refer to the bit structure of the words in the fictitious computer. 

The intent of Parts I and II is to give the reader enough background 
so that he can pick up the reference manual of an actual computer and 
start programming. Learning to program the fictitious computer in this 
book is likely to help rather than hinder learning some actual machine, 
since most computers are very similar to each other. There is, of course, 
no substitute for experience, and the reader should, if he possibly can, 
gain some firsthand experience by programming a computer. A reader 
with a particular machine at hand can easily translate the sample pro- 
grams of this book into the language of his machine, once he has grasped 
the principles set forth in Chap. 3. 

It is possible to use computers effectively without understanding 
binary numbers or any other details of the machines. The programmer 
can work strictly within the bounds of a programming language, leaving 
all details of bit-by-bit processing to the professionals who prepare the 
languages. Those who wish to avoid binary matters, either permanently 
or temporarily, can simply omit Parts II and IV, though they will have 
to disregard occasional remarks about bits in Part III. But by remaining 
ignorant of binary affairs, the new programmer will find himself at a 
disadvantage in communicating with experienced programmers and 
computer operators when he needs help. He also forfeits the opportunity 
to understand the beautiful simplicity of the machines and must remain 
forever in awe of their mysterious powers. Learning how a computer 
works is useful, intellectually satisfying, and not very difficult. There is 
one exception. The processes by which computers take in and put out 
information are ugly and intricate beneath the surface. Here the novice 
is urged to curb his curiosity and use existing programs. 

For clarity and brevity I have not hesitated to impute human proper- 
ties to computers. Phrases like “The computer knows . . .” and “The 
programmer must tell the computer . . .” are convenient abbreviations 
for long-winded mechanistic statements and seem so natural that I 
cannot avoid them. They are common phrases in the everyday language 
of information processing and have exact operational definitions in each 
specific context. Any attempt to remove all anthropomorphic phrases 
from an account of digital computers is doomed either to failure or to 
low scores in readability. 

My colleagues in the Psychology Group at the Lincoln Laboratory, 
Massachusetts Institute of Technology, contributed many helpful 
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suggestions and criticisms. Warren Torgerson, Douwe Yntema, R. T. 
Mitchell, Alice Wolf, Laura Klem, Benjamin White, William Harris, Keith 
Smith, and Carol Chomsky caught many errors, improved my exposition, 
and gave me many good ideas that are now part of the book. I am also 
grateful to William McGill, Allen Newell, Herbert Simon, George Miller, 
Donald Taylor, Carl Helm, Robert Gibson, Edward Johnson, and Marvin 
Minsky for their critical reading of parts of the manuscript. The Baseball 
program, described in Chap. 13, is the joint work of Alice K. Wolf, Carol 
Chomsky, Kenneth Laughery, and myself; it was first conceived by 
Frederick C. Frick, Oliver G. Selfridge, and Gerald P. Dinneen, whose 
continued support is gratefully acknowledged. 

I wish to thank Shirley Kimber, Judith Chandonnet, and Peggy Kmiec 
for their careful typing of the manuscript, members of the Lincoln 
Laboratory Publications Group for their help in preparing the figures 
and reproducing the drafts, and my wife Becki for her cheerful help in 
editing, proofreading, and checking. 


Bert F. Green, Jr. 
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Introduction 


f lis first electronic digital computer was built during World War II 
and was publicly unveiled in 1946. Since then, digital computers have 
evolved and expanded so rapidly that today they are a thousand times 
more powerful than their early predecessors, and they are still growing. 
The number of computers in operation is also growing rapidly. Today 
most large universities, businesses, and independent laboratories have 
a digital computer of some sort, and many have large high-speed 
computers. 

Digital computers have turned out to be useful in many more ways 
than their originators imagined. The machines were developed primarily 
for solving long and complicated mathematical problems involving 
millions of separate calculations. But the same machine that will 
diligently perform the tedious calculations of atomic physics and aero- 
dynamics will also do the tedious bookkeeping and inventory chores of 
the business world and the tedious statistical analyses of behavioral 
science. Moreover, many problems that seem essentially nonnumerical 
can be rephrased to take advantage of the computer’s untiring capacity 
for manipulating digits. 

Because computers spend much of their time calculating complicated 
mathematical formulas, it is commonly thought that computers are 
“giant brains” and can be mastered only by mathematical wizards. In 
fact, computers are “giant clerks’ and can be instructed by anyone who 
understands the series of operations that he wants performed. Most 
digital computers have a built-in capacity for only some very elementary 
operations, like adding, subtracting, and comparing. To achieve more 
complex results, the user must combine these operations by instructing 
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the computer to perform its elementary operations in a sequence that is 
appropriate to his problem. A mathematician can specify a sequence of 
operations that enables the computer to solve a differential equation; 
an accountant who never heard of a differential equation can instruct 
the same computer to act as a bookkeeper; and a psychologist can 
instruct the computer to produce random lists of verbal stimuli for 
experiments in human memory. A computer by itself can do nothing, 
but the same computer with suitable instructions can solve equations, 
keep books, list stimuli, predict the weather, or prove theorems in plane 
geometry. 

A sequence of instructions suitable for solving a particular problem is 
called a program, and the process of devising such a sequence is called 
programming. Programming requires precision and logical clarity but 
does not demand special mathematical training. The programmer needs 
to know only the mathematics inherent in his problem. Programming 
skill appears to be related to quantitative ability because the good 
programmer, like the good mathematician, is adept at translating 
problems into the precise symbols and relations of his trade. Yet the 
basic rules of programming are straightforward and easily learned. 
Anyone who applies the rules with intelligence, thoroughness, and a 
little ingenuity can devise programs for the powerful new machines. 


1-1 Historical note 


The automatic digital computer is a phenomenon of the twentieth 
century. It is the product of advanced skill in electrical engineering, and 
of contemporary willingness to devote a great deal of money and man- 
power to scientific development. That such a machine could be built in 
the nineteenth century would seem unbelievable, and yet a mechanical 
equivalent of today’s electronic digital computers was very nearly built 
a century ago by a British mathematician named Charles Babbage. In 
1822 Babbage demonstrated a small working model of a difference engine 
which could do little more than add and print results, but with which 
some mathematical tables could be computed. While engaged in building 
a large difference engine, he conceived the idea of an analytic engine 
that would perform all kinds of arithmetic operations. The analytic 
engine was, in fact, a universal computer, sufficiently complex to perform 
any finite computation. 

The analytic engine, like the difference engine, was too ambitious for 
the mechanical skills of Babbage’s time. Babbage not only designed the 
machine but was forced by the primitive state of contemporary engineer- 
ing to design better gears. Moreover, his analytic engine was to have 
accommodated 1,000 fifty-digit numbers and would have required an 
assemblage of gears and levers weighing several tons, Had Babbage been 
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more modest in his aspirations, he might well have succeeded and the 
history of digital computers would have started a century earlier. As it 
happened, Babbage failed to build his machines and his ideas were 
forgotten, to be reinvented in the twentieth century. 

Computer history really begins with the development of punched-card 
business machines at the beginning of the twentieth century. In 1890 
Hollerith proposed using punched cards in conjunction with electro- 
mechanical relays to perform the simple additions and sortings needed 
by the U.S. Bureau of the Census. In the next 50 years, business ma- 
chines based on this principle were expanded and perfected by such 
companies as International Business Machines (IBM) and Remington 
Rand. Punched-card business machines can do little more than add and 
subtract, and while ingenious ways were found to use them for more 
complex calculations, they are basically oriented toward business ap- 
plications. In the late 1930s and early 1940s Aiken at Harvard and 
Stibitz and others at the Bell Telephone Laboratories designed special 
machines for the automatic calculation of complex mathematical func- 
tions. These machines were fashioned out of electromechanical relays, 
which are the components found in business machines and telephone 
switching equipment. Aiken’s relay computer, MARK I, built in co- 
operation with the IBM Corporation, was demonstrated in 1944. This 
computer probably deserves to be called the first modern automatic 
digital computer. 

World War II brought great technical advances in the electronics 
industry and also great pressures for swift automatic calculations. A 
completely electronic computer now appeared both feasible and neces- 
sary. The doubtful reliability of electronic components had been the 
main obstacle, and by the end of the war, industry had mastered the 
difficulties. The first computer to be made entirely of the swift new 
electronic parts was the Enrac, built by Mauchly, Eckert and their 
collaborators at the University of Pennsylvania in 1946. The same 
group, with the advice of John von Neumann, completed the Epvac in 
1949. The Epvac had much more internal storage capacity than the 
nrac and was the first digital computer to store its program internally 
rather than in a plugboard. The first commercial computer, Unrvac I, 
was also built by this group and appeared on the market in 1951. 

During the postwar years the Massachusetts Institute of Technology 
(MIT) was building Whirlwind, which began operating in 1949 and was 
in its day the fastest digital computer. Other experimental computers 
were being built in England. In 1952 and 1953 von Neumann’s Institute 
for Advanced Study (IAS) computer and its copies appeared, including 
the Orpvac, Inu1ac, JouNnNrAC, and others. Unrvac and its predecessors 
used mercury delay lines as storage devices, and these devices proved 
temperamental. Whirlwind, the [AS computers, and some of the English 
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computers used electrostatic storage tubes, which were also tempera- 
mental and subject to long-term wear. In 1952 Forrester at MIT 
developed the magnetic core as a storage device and installed magnetic 
cores in Whirlwind. This was so successful that nearly all high-speed 
computers designed after that year have used magnetic cores for storage. 

In 1953 IBM completed its first commercial high-speed computer, 
the 701, with electrostatic storage. After making several of these, the 
model was discontinued in favor of the 704, with magnetic cores. At the 
same time several other commercial organizations began making high- 
speed computers, and computer development became a commercial as 
well as an academic enterprise. 

The postwar years saw a parallel development of the medium-speed 
computers, which use a mechanically rotating magnetic drum for stor- 
age. Various commercial computers of this type became available in the 
early 1950s, including the prevalent IBM 650. 

Computer development is still continuing at a fast pace today. The 
transistor is reducing the power requirements for large computers and 
increasing their reliability; it is also allowing much more compact units. 
High-speed storage units are being increased in capacity—at least two 
computers have a capacity of over 2 million binary digits. High-speed, 
versatile input-output equipment is improving the match between the 
computer and its environment. The ability to have several programs 
running concurrently, in a time-sharing fashion, permits the efficient use 
of computers in situations involving ongoing interactions between men 
and machine. 

The future will bring even more powerful machines. Computers with 
more storage capacity, faster operation, and more flexible logical struc- 
ture are all on the horizon. As more scientists learn how to use these 
machines, the growth of problems requiring solution will doubtless keep 
pace with the technological advances. 


1-2 Calculating with digits 


The computers discussed in this book are called digital because they use 
digits to represent information. In doing numerical calculations, a digital 
computer uses strings of digits to represent numbers, just as we use 
numerals in doing arithmetic with pencil and paper. Digital computing 
is done by following prescribed rules for combining digits, such as the 
familiar rules of arithmetic learned in grade school. One of the earliest 
digital calculators was the abacus, which has a rod for each digit; the 
value of the digit is represented by a pattern of beads on the rod. 
Numbers are added by changing the bead pattern according to pre- 
scribed rules. Desk calculators are elaborate mechanical abacuses with 
cogwheels in place of beaded rods. Connections between cogwheels are 
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arranged so that the rules of arithmetic (such as the carries in addition) 
are followed automatically. The electronic digital computer has elec- 
tronic circuits that behave in the same stepwise fashion as the cogwheels 
and the beads. 

Any number can be represented as a string of digits. Numbers that 
cannot be represented exactly, such as 7 or 14, can be represented as 
accurately as necessary by using enough digits. For 2-place accuracy, 
14 = 0.33, while, for 10-place accuracy, }4 = 0.3333333333. Moreover, a 
continuous, smooth function like y = 4x — 2? can be represented 
digitally by a sequence of numbers in the same way that a continuous 
distribution function can be repre- 
sented by a bar graph, illustrated 
in Fig. 1-1. With enough numbers 
the correspondence between the 
continuous and discrete forms can 
be as close as required. 

Negative numbers can be repre- 
sented as easily as positive num- 
bers. An extra digit is attached 
to every number, positive or neg- 
ative, to indicate the sign of the 
numbers. Needless to say, the rules 
for sensing and manipulating the 
sign digits are different from the 
rules for numerical digits. 

Special rules are also needed 
for the decimal point, which occurs Fis- 1-1. The discrete approximation 
explicitly in mixed numbers like of a continuous function. The closer the 

; site ‘ bars are spaced, the more nearly accu- 
10.25 and implicitly in all numbers. pate is the approximation. 
Readers familiar with desk calcula- 
tors will appreciate the fact that the location of the decimal point can be 
an annoyance in digital calculations. Some computers require the pro- 
grammer to keep track of the decimal point or to include specific pro- 
visions in his program for locating it. Many of the newer computers now 
have an optional feature that allows the computer to keep track of the 
point automatically, using some additional digits to code its location. 

Binary digits. Man has 10 fingers, and it is generally agreed that 
our decimal number system stems from this biological circumstance. If 
an octopus should develop a need to count, he might find it convenient 
to count by 8s, using an octal number system. Digital computers happen 
to find it convenient to count by 2s, using a binary number system. A 
binary digit has only two alternative values, 0 and 1, corresponding to 
an electric light that is either off or on or to a switch that is either open 
or closed, Because a binary digit has only two possible values, it can 
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readily be handled electronically and forms the basis of all electronic 
digital computers. Some computers disguise their binary nature by using 
intermediate units that count up to 10 by 2s and then perform all 
computer operations with the intermediate units. Such machines are 
called decimal computers and have the outward appearance of counting 
by 10s, though at heart they are binary. Other computers, including 
most scientifically oriented computers, make no bones about it—they 
are out-and-out binary computers. They handle information in the form 
of strings of binary digits, that is, ls and 0s. Data must be translated 
into binary digits by utility programs before they can be stored in the 
computer. ; 

Strings of binary digits form binary numbers, just as strings of 
decimal digits form our familiar decimal numbers. The binary and 
decimal number systems are completely analogous. The string of 
decimal digits 125 stands for 100 + 20 + 5. This can also be written as 
1x 10? +2 x 10'+5 X 10°, indicating that the digits are multipliers 
of successive powers of 10. In a binary number the digits multiply 
successive powers of 2. Thus the binary number 101 stands for 1 X 2’ 
+0 xX 2!+1 X 2°, which is equivalent to the decimal number 5. 
Additional details of the binary number system are covered in Chap. 6. 

Although a prospective user of a binary computer should understand 
binary numbers, he need not actually use them. He can avoid all binary 
details by taking advantage of special translation programs written by 
professionals. The user can work entirely in the familiar decimal system, 
relying on the translation programs to turn everything into binary 
notation for the computer and to reinterpret the computer’s results in 
decimal notation. Only in very unusual circumstances must the user 
pay attention to the digital representation within the machine. 

Analog computers. Calculating with digits is not the only way to 
calculate with machines. Analog computing is done by combining 
quantities rather than digits. In its simplest, nonelectronic form, analog 
addition may be accomplished by selecting sticks with lengths pro- 
portional to the numbers being added and placing them end to end to 
form a single combined length. The slide rule uses a more sophisticated 
technique, with numbers being represented by lengths proportional to 
their logarithms. Numbers can then be multiplied on a slide rule by add- 
ing their corresponding lengths. Electronic analog computers generally 
use electrical voltages or other electrical quantities as analogs for 
numbers and contain electronic circuits to add, multiply, and otherwise 
combine these quantities. Very complex calculations can be performed 
by connecting these circuits in simple networks. 

Programming an analog computer requires a fair degree of electronic 
sophistication, The programmer must understand how combinations of 
circuits will behave, and he must also know how to translate mathe- 
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matical expressions into circuit combinations. The mathematics of signal 
analysis is often indispensable in preparing an analog program. The 
digital-computer programmer, on the other hand, needs no knowledge 
of the electronic features of his machine. All he needs is a knowledge of 
the machine’s logic and, if his problem is mathematical, some knowledge 
of numerical analysis. In short, analog programming is mainly an 
exercise in electronics and signal analysis, while digital programming is 
mainly an exercise in logic, with perhaps a little numerical analysis. 

One advantage of digital computation is its potentially greater ac- 
curacy. There is essentially no limit to the number of digits in a digital 
calculation, whereas the difficulty of controlling the physical variable 
limits the accuracy of any analog device. If a particular problem required 
numbers from 0 to 99, a digital computer might use two decimal digits to 
represent the numbers, while an analog computer might measure the 
total available range of voltage on a scale of 100 intervals. For a problem 
requiring numbers from 0 to 999, the digital computer would use three 
digits, while the analog computer would measure the voltage range on 
a 1,000-interval scale. For problems requiring greater accuracy, the 
digital computer can simply add more digits, but there is a limit to the 
number of voltage intervals that can be accurately measured in the 
analog computer. Accuracy better than 1 part in 1,000 is very difficult 
to achieve in analog computers. A related distinction between analog 
and digital machines is the kind of errors made. When an analog com- 
puter errs, the number in error is likely to be nearly correct, for example, 
175 instead of 173, since the error usually stems from inaccuracies in 
voltage control. A digital machine, on the other hand, is as likely to make 
an error in one digit as another, for example, 273 instead of 173, since the 
error probably results from a bead or cog slipping somewhere. 

Analog computers are well suited to some special types of problems— 
notably those in which the inputs and outputs are complicated functions 
of time. Hydrodynamic and aerodynamic problems are often of this type, 
involving partial differential equations relating movements to time. In 
psychology, the analysis of human performance in continuous tracking 
tasks requires comparing two inputs, stimulus and response, that are 
continuous functions of time. In studying compensatory tracking a 
rudimentary analog computer is a necessary part of the experimental 
apparatus, since the stimulus to the subject is the difference between his 
response and the original input signal being tracked. 

lor most applications, analog computers are losing popularity, since 
their digital competitors are much more versatile. A digital computer can 
perform any computation that an analog computer can, while the con- 
verse is not true. Any continuous analog variable can be represented 
digitally by a sequence of discrete numbers, and with enough numbers 
the agreement of the analog and digital forms can be as close as required. 
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Asuitable sequence of instructions can cause a digital computer to mimic 
the behavior of each analog circuit, and the complete analog-computer 
program can be matched by properly combining the digital instructions 
into a digital-computer program. The digital process may sometimes be 
slower than its analog counterpart, but the net result will otherwise be 
the same. Moreover, a digital computer can even by made to behave as 
if it were an analog computer by equipping it with devices to convert 
the input from analog to digital form and to convert the output from 
digital to analog form. 


1-3 Digital computers as information processors 


The fact that a digital computer calculates by manipulating digits is 
significant, because digits can be used to stand for many things besides 
numerical quantities. In fact digits can be used to represent any infor- 
mation whatever. Or, to put it another way, all information can be 
represented by numbers, and hence by digits. 

Information theory teaches that any item of information can be 
viewed as a choice among a fixed number of alternatives or as a finite 
sequence of such choices. Information can then be represented digitally, 
or coded, by simply numbering the alternatives. Each item of information 
is represented by the number that corresponds to the selected alternative. 
Numbers used to signify alternatives in this way do not have any quanti- 
tative meaning. Like telephone numbers, they are merely convenient 
identifying symbols. 

As a concrete example, consider translating English text into digits. 
One simple way would be to assign a pair of digits to each letter in the 
alphabet, from A = 01 to Z = 26. Note that the digital symbol for A is 
not the number 1 but the pair of digits 01. To complete the code, other 
digit pairs must be assigned to the various punctuation marks, such as 
00 for space, 27 for period, etc. Then the sentence “Go by cab.” would be 
encoded into the string of digits, 07150002250003010227. This code is 
easily remembered because the digit pairs are assigned in numerical 
order to the letters in alphabetical order, but any other unique assign- 
ment of digits to letters would be equally valid. In some applications 
it might be convenient to code text by assigning digits in numerical 
sequence to the keys of the standard typewriter keyboard. A quite 
different procedure would be to number the successive words in some 
standard dictionary. The text to be encoded would then have to be re- 
stricted to words contained in the dictionary. Which code is preferable 
depends on the circumstances of the particular problem. 

Computer programs for processing coded information must incorpo- 
rate, either implicitly or explicitly, the rules by which the information is 
coded into digits. The programs are composed of computer operations 
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such as comparing digits, counting digits, and rearranging digits. 
Elementary computer operations are sufficiently general so that combi- 
nations can be devised to process any coded information in any specified 
way. 

Since a digital computer can process any kind of information with an 
appropriate program, a computer is more than an automatic calculating 
machine. In fact, the terms information-processing machine and electronic 
data-processing machine are often used today to emphasize the broader 
applications of the machines. It is true that the design of many com- 
puters is slanted toward mathematical calculation, so that programming 
nonnumerical processes is more difficult than programming numerical 
calculations. However, a variety of nonmathematical programs can 
generally be found to facilitate the handling of coded information. 

Universal Turing machines. Digital computers, or information- 
processing machines, are concrete instances of a general class of theo- 
retical machines that may be called universal symbol manipulators. The 
significance of the term arises from a remarkable theorem of Turing 
(1936) to the effect that any finite symbol-manipulation process can be 
performed on a machine of sufficient complexity. Turing defined a 
machine that has come to be called a universal Turing machine and 
showed that his universal machine could perform any calculation that 
could possibly be calculated by any finite process. The universal machine 
thus represents an upper limit on the necessary complexity of a 
computer. Later workers have shown that there are many different 
equivalent definitions of the universal machine. To show that any 
particular machine is universal, one must show that it can be made to 
behave in accordance with any one of the equivalent universal machines. 
In particular, it is easy to show that any general-purpose digital com- 
puter is a universal Turing machine. The only qualification is that the 
storage capacity of the computer must be sufficient for the problem. 
Turing machines are discussed further in Chap. 15. 

The fact that every general-purpose digital computer is a universal 
Turing machine implies that any digital computer can be programmed 
to do anything that any other digital computer can do, if the appropriate 
input and output mechanisms are available, and if there is sufficient 
storage capacity. Since even the simplest digital computer is universal, 
differences among machines imply differences in efficiency, not differ- 
ences in ability. The limitation of capacity is also a question of efficiency 
in the sense that most problems that exceed the machine’s capacity can 
be broken into segments each small enough for the computer. 

Turing’s theorem also implies that a digital computer can be pro- 
grammed to solve any problem that can be resolved into a finite sequence 
of symbol manipulations. This statement seems tautological, but in 
practice it means that any information-processing problem that has been 
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stated precisely, unambiguously, and in detail can be programmed. If 
no program can be found, then (1) the programmer’s ingenuity is faulty, 
or (2) the problem must be stated more precisely, or (3) the problem 
implies an infinite recursion and cannot be solved by finite symbol 
manipulations. The last is very rare; the first two are the main stumbling 
blocks. This is not to say that the programmer is stupid if he fails to find 
a program. Many problems lie beyond our present scientific accomplish- 
ments. No one knows how to program a computer to recognize faces or 
even to match fingerprints. Yet both processes are easily done by human 
beings. If one takes the view that all human processes are deterministic, 
it follows that these processes can be programmed for a computer. The 
difficulty lies in making a detailed, precise definition of the processes. 
We do not know precisely what we mean by recognizing faces or matching 
fingerprints. Turing’s theorem focuses our attention on the processes, by 
denying the easy excuse that the machines themselves are not sufficiently 


complex. 


1-4 General characteristics of computers 


Electronic digital computers are novel devices. The way in which they 
operate bears little resemblance to more familiar machines. Some of the 
major aspects of computer operation are listed here as an introduction to 
the more detailed discussion in later chapters. 

General purpose. The vast majority of digital computers have 
been built for a variety of possible uses. Their elementary operations are 
sufficient for any possible calculation and have been chosen for maxi- 
mum flexibility. Their input devices will accept data in a variety of 
different formats, and their output devices are equally general. All 
specialization is achieved through programming. These machines are 
called general-purpose digital computers to distinguish them from special- 
purpose machines that have been built for restricted use in a specific 
application. This book deals only with general-purpose machines. 

Speed. Electronic digital computers operate at a very high speed. 
Computers that can perform 1,000 operations per second are considered 
to have only medium speed. High-speed machines perform up to 200,000 
operations per second. A high-speed computer can calculate a table of 
trigonometric functions with over 1,000 entries in less than 2 seconds. It is 
a sobering experience to put a small program into a computer and to find 
that the computer has completed the calculations in the time that your 
finger is releasing the START button. This incredible speed is achieved 
by electronics—there are no moving mechanical parts in the processing 
units of high-speed computers.. The slower speed of the ‘‘medium” 
computers is the result of one or two moving mechanical parts. 
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Information storage. Modern computers operate so fast that they 
cannot efficiently receive information one item at a time as the need 


_ arises. Rather, all the information must be stored in the computer at the 


beginning of a problem, after which the computer can operate on the 
information at its own speed. Consequently, if the computer is to work on 
large problems, it must have the capacity for storing large quantities of 
information. One of the major parts of a digital computer is its memory, 
or storage unit. The memory is usually capable of storing thousands of 
digits, and some memory devices can store millions of digits. The 
information in memory is not stored permanently. It can be easily 
erased, so that new information can be stored in place of old information. 
In fact, the very act of storing new information generally causes the 
previous information to be erased. Thus the memory can be used over 
again for each new problem. 

Stored programs. The computer’s speed, which argued against 
presenting information one item at a time, also argues against giving the 
instructions one by one as the calculation proceeds. The entire program 
of instructions must be placed in the machine’s storage, along with the 
data, at the start of the problem. After the program has been stored 
initially, the machine can carry out the sequence of instructions auto- 
matically, at its own speed. Since the storage unit can store only digits, 
the instructions in the program must be translated into digital form. 
Instructions can easily be coded digitally, since each possible machine 
operation can be given a number. 

Self-modification; efficiency through repetition. The idea of 
storing the program is very powerful. Apart from the speed and efficiency 
gained, the computer acquires the facility to modify its own program, 
because the instructions are in the storage unit and can be operated 
upon, just as the stored data can be altered. The self-modifying program 
is the key to automatic data processing. If the programmer literally had 
to write down every instruction that the computer was to perform, he 
might as well do the calculation by hand. But the self-modifying feature 
of the digital computer solves the dilemma. When the same sequence of 
instructions is to be performed on many pieces of data, a single basic 
sequence is written, along with some instructions to modify the basic 
sequence so that it operates on a different datum each time. A digital 
computer thus becomes very efficient at repetitive work. In an efficient 
program almost all the instructions are carried out many times. For 
example, adding a column of 40 numbers does not require 40 ADD 
instructions in the program. Instead, a basic sequence of instructions is 
formed, consisting of a single ADD instruction and one or two modifying 
instructions; this loop of instructions is traversed 40 times to add the 
column, The same loop could be used for a column of any specified 
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length. The loop principle works at all levels of programming. To find 
the means and sigmas of many distributions, a sequence of instructions 
that finds the mean and sigma of one distribution is made into a loop 
which is traversed once for each distribution to be processed. This 
distribution loop will be very likely to include subloops—such as the 
addition loop mentioned above. An efficient program is a whole nest of 
loops within loops. 

Repetitive operations are as indispensable to computers as their speed. 
The combination of speed and repetition compensates for the ele- 
mentary nature of the computer’s repertoire of built-in operations. 
Speed alone is no help. A list of 1,000 instructions with no repetitions 
can be performed in less than 1 second, once the instructions are stored 
in the machine, but preparing the instructions and storing them in the 
machine are time-consuming. And repetition alone is no improvement 
over a clerk with a desk calculator. The clerk at least has the advantage 
of being able to respond to more complex instructions than the computer. 

Contingent programs. The ability to execute loops repetitively 
implies a further computer ability—decision making. At the simplest 
level, the computer must be able to decide whether or not a given loop 
has been repeated the specified number of times. If not, the loop is 
repeated again. If so, the computer goes on to the next part of the 
program. Such contingent behavior can be made more complex. The 
computer can be set to take one of several different courses of action 
depending on the results of previous calculations. Of course, all the 
contingencies must be foreseen, so that the program can provide for 
every alternative. 


1-5 Programming 


Programming a digital computer is an exacting task, because the ma- 
chine is such a relentless follower of instructions. It does just exactly 
what it is instructed to do, showing no common sense at all. Every 
possible contingency that might arise in the course of the calculation 
must be foreseen and provided for. If the result of a calculation is a 
number too large for the computer, or if negative numbers crop up where 
they should not, chaos may result unless the programmer has anticipated 
the possibility and has made provisions for the computer to detect the 
situation and take corrective action. 
Programming also requires careful attention to details. The com- 
puter’s built-in repertoire of operations is so elementary that a process 
must be dissected into very small pieces in order to be programmed. 
The instructions for a computer must be considerably more detailed 
than those for a human clerk. Nevertheless, when a process is clearly 
understood, it can be programmed readily, Indeed, the act of program- 
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ming may uncover some “holes’’ in the original specifications of the 
process. 

Programming aids. Precision and detailed analysis, although some- 
times beneficial, can be onerous. Happily, the burden can often be 
eased by taking advantage of previous work. An extremely lucky pro- 
grammer may find that his program has been written for his computer 
by someone else and has been filed in a program library maintained by 
the computer installation. Normally he will not be so lucky, but the 
library is sure to have a number of useful programs or subroutines that 
he can use as parts of his program. In effect, subroutines greatly increase 
the computer’s primitive ability. For example, in arithmetic calcu- 
lations, the computer by itself can only add, subtract, multiply, and 
divide. But with subroutines the computer can take square roots and 
compute logarithms, exponentials, trigonometric functions, and other 
more complicated functions. 

Subroutines from the computer’s library are also useful in program- 
ming the input and output phases of a complete computer program. The 
process of putting data into the computer involves translating from the 
user’s notation, such as decimal numbers, into the computer’s digital 
representation and also transferring the information from some external 
form, such as punched cards, into the computer’s storage unit. Later, 
when the results are to be put out, they must be translated into the user’s 
symbols and transferred from the computer’s storage to an external 
form, such as a printed sheet. Transferring and translating data are 
standard processes for which general-purpose subroutines are always 
available. To prepare the input and output phases of a new program, the 
programmer need not understand the detailed workings of the com- 
puter’s elementary input and output operations; instead, he must under- 
stand how to use the appropriate library subroutines. 

In recent years it has become evident that much of the detailed 
analysis involved in programming is so straightforward that it can be 
done by the computer itself. Elaborate programs called compilers have 
been devised that will accept processes specified in some precise but 
convenient notation and that will produce a detailed computer program 
matching the specifications. The precise notation recognized by a 
compiler is called a programming language. The best-known language 
is ForTRAN, which allows mathematical programs to be written in a 
form of algebraic notation similar to that used in mathematics texts. 
Many computers have compilers that will translate programs written 
in orTRAN language into a sequence of instructions for that computer. 
Other languages exist for expressing business processes and nonnumeric 
symbol manipulations. These languages do not lessen the need for pre- 
cision, but they protect the programmer from many of the tiresome 
details of the machine, Programming languages are discussed in Chap. 5. 
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Programming costs. More time and effort go into preparing a 
computer program than most people suspect. Not only does writing 
the program take time, but the process of eliminating all the errors 
from the program can be very lengthy. There are so many opportunities 
for errors in programming that mistakes seem inevitable, and the first 
results produced by a computer program are likely to be nonsense. 
Finding the errors is a process of deduction, based on the discrepancies 
between the expected results and the actual results. Special programs 
are generally available to aid the programmer in this process, but the 
job is still difficult. 

In short, using a computer to solve a problem is not to be undertaken 
lightly. Any problem that can be done manually in a few man-days 
should probably not be done with a computer unless the program can 
be used over and over again or unless part of one’s motivation is to learn 
about computers. Using a computer is always a major enterprise, no 
matter how simple the problem. On the other hand, a computer can make 
short work of tasks that simply could not be done in any other way, and 
the pleasure of getting such swift results can easily offset the pain of 
preparing and ‘“‘debugging”’ the program. 


1-6 Digital computers in the behavioral sciences 


The universal nature of computers suggests a wide variety of appli- 
cations. Behavioral scientists have already taken advantage of com- 
puters, not only to calculate extensive statistical analyses, but also to 
produce stimuli for experimental research, to construct and test new 
models of behavior, to study the human problems that arise in complex 
military and business systems, and to process the volumes of data 
generated by huge psychometric testing programs. Every branch of 
behavioral science is profiting either directly or indirectly from these 
versatile machines (see Wrigley, 1957). 

The most frequent use of computers in psychology has been in the 
statistical analysis of experimental data. The large variability generally 
encountered in such data requires the use of sophisticated statistical 
techniques to determine the influences of the experimental variables. 
The investigator’s interest in the interrelations among many concurrent 
variables often leads him to the application of complicated methods of 
multivariate analysis. Before computers, an investigator would often 
think twice about investing all the necessary man-hours to produce the 
results of one of these analyses, but now a computer can do the job in 
a few minutes. Preparing the data for the computer still takes time, but 
the investigator is much more willing to spend this time, because of 
the increased power of the computer. He can try several methods of 
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analysis and investigate many special side issues that would have been 
bypassed in a manual analysis. 

Some methods of analysis are so complicated or so tedious that they 
cannot reasonably be applied without a computer. Techniques that have 
gathered dust in journals or have been only mathematical showpieces 
are now suddenly practical. New analytical techniques can now be 
developed without the fear that their application will be too time- 
consuming to be worthwhile. On the other hand, this newfound freedom 
implies additional responsibilities. New techniques must be subjected 
to practical test before being judged acceptable. An elegant mathematical 
formula is no longer enough. 

Experimental psychologists are finding computers useful, not only in 
analyzing results, but also in preparing stimuli for experiments and even 
in conducting the experiment. Experiments involving huge amounts of 
stimulus material can now be contemplated seriously instead of being 
filed in a dream basket. Computers can easily produce thousands of 
randomized lists, thousands. of pictures each containing thousands of 
randomized elements, and thousands of other needed items, all according 
to precise specifications. The computer can even be used to run the ex- 
periment. Running experiments by a computer is usually uneconomical, 
because the machine is so much faster than the human or animal 
subjects. But when complex feedback arrangements must be made after 
every response, or when many subjects can be run simultaneously, or 
when the experiment is sufficiently complicated, computer control 
becomes economically feasible. 

Applied psychologists are also finding a variety of uses for computers. 
The many bookkeeping and statistical chores involved in large-scale 
psychometric testing programs are being turned over to computers, and 
it is likely that some rudimentary interpretation of results from intelli- 
gence tests will soon be automated. Scoring and interpreting personality 
tests are much more difficult problems, but computers are destined to 
play a significant role in personality assessment. The clinician can also 
expect to take advantage of developments in the field of information 
retrieval to help him find precedents and prognoses for his current cases. 
lducators are already using simple teaching machines to great advantage 
in developing basic skills in children. With a computer controlling the 
presentation of material, the course of study and practice can be tailored 
to the needs of the individual student. 

One of the most interesting applications of computers in psychology 
is the testing of theories and models of behavior. Some models are 
simple in theory but lead to ugly mathematical complexities. The ideas 
may be reasonable and the original mathematical representation may be 
faultless, but mathematical tools needed for analyzing the model may be 
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lacking. In this situation a computer can be used to test the model 
empirically, using data generated from random numbers according to 
theoretical specifications. Other models, particularly in the area of con- 
cept formation and problem solving, cannot conveniently be represented 
by mathematical equations. The models specify a sequence of processes 
containing many contingencies and can best be realized by a computer 
program. While such information-processing models can be tested by 
hand by carrying out the indicated processes manually, they can be 
tested more easily and more thoroughly with a computer. 

From this brief, incomplete list it is clear that computers are valuable 
assets in psychological research. But they are also potential dangers. 
Psychologists have a history of being fascinated by the tools of their 
trade, and computers are immensely fascinating. Like the investigators 
who spend inordinate amounts of time and energy in building experi- 
mental apparatus or devising elegant statistical techniques as a sort of 
escape from the main business of science, some psychologists can be 
expected to be lured by the challenge of computer programming. For 
many, programming a computer is a giant game, a stimulating battle of 
wits with its own built-in gratifications. Of course, the only sure way to 
avoid the lure is to avoid programming, but the possibilities of a com- 
puter cannot be appreciated fully without firsthand experience. The 
best strategy is to recognize the danger and plunge ahead. 
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Functional description of digital computers 


(a digital computers that have been designed for a 
wide variety of possible applications are basically alike, although 
each differs from the others in many details. The following description 
will concentrate on the similarities and will largely ignore the differences. 
Ultimately, the programmer must 
face up to the idiosyncrasies of the 
machine he proposes to use, but a 
broad-brush rendition will do for an 
introduction. 

All general-purpose digital com- 
puters consist of four major sec- 
tions, as shown in Fig. 2-1: some 
input-output equipment for trans- 
ferring information into and out of 
the computer, a storage unit in which 
data and instructions can be held, a processing unit that performs various 
operations on the stored data, and a control unit that coordinates the 
other units so that the desired sequence of operations is performed. 
These functional units may not correspond to the apparent physical 
units of the computer, as shown in Fig. 2-2. A visitor to a computer 
installation might find many more than four cabinets, or perhaps fewer. 
And some of the cabinets that he sees may contain such units as power 
supplies and cooling systems that are vital to the machine but of no 
concern to the programmer, But no matter how the computer is pack- 
aged, its logical design has the four sections named above. 
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Fig. 2-1. Functional units of digital 
computers. 
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Fig. 2-2. A typical computer installation. (Photograph of an IBM 7090 computer 
installation, courtesy of International Business Machines Corporation. Used with 
permission.) 


2-1 Input-output 


The user communicates with the computer by means of the input and 
output equipment. The input devices feed data and instructions to the 
computer, and sooner or later the computer sends back its results 
through the output devices. Input and output techniques vary widely 
among computers. Punched cards, punched paper tape, and magnetic 
tape are the most prevalent input media, while the results of a calcu- 
lation are usually recorded on magnetic tape, punched into cards or 
paper tape, or printed directly. 

Input and output processes involve a number of auxiliary machines 
not connected to the computer. Cards must be punched and magnetic 
tape must be recorded before the computer’s input mechanisms can 
accept them. Likewise, the output of the computer, when in the form 
of cards or tape, must be turned into a printed record for the user. 
Machines that perform these auxiliary chores are called off-line ma- 
chines, to distinguish them from the on-line machines that are connected 
directly to the computer. The reason for distinguishing between off-line 
and on-line machines has to do with efficiency. Input and output 
mechanisms, being electromechanical, are much slower than the com- 
pletely electronic storage, processing, and control units. To conserve 
valuable computer time, the on-line input and output processes must 
be accomplished swiftly, by using the most, efficient mechanisms, As 
much as possible of the input and output processing must be done off 
line, 
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The separation of off-line and on-line functions is a step toward 
efficiency but the computer must still twiddle its thumbs when the on- 
line input-output equipment is operating. A high-speed computer can 
perform over 10,000 internal operations in the time it takes one card to 
be read in and over 500 operations in the time the same amount of 
information can be read in from magnetic tape. As a further step toward 
efficiency, the newest computers have been given the ability to read data 
into memory at the same time that internal calculations are proceeding 
with other data already in the computer. The new computers can also 
read results out of memory concurrently with other internal processing. 
In fact, on some new computers several input-output processes can 
proceed concurrently, along with the internal processing. 

Mechanisms. A wide variety of input and output mechanisms will 
be found on different computers. Each individual computer generally 
has more than one mechanism for input and output, since no single 
mechanism is completely satisfactory for all applications. Some interest- 
ing details of the common mechanisms are described below. These 
details are not crucial to a general understanding of how computers work. 

Punched cards are the most widely used input media. Punched-card 
machines were in use long before automatic digital computers were 
developed and have been adapted to their new role with essentially no 
change. Standard key punches are used to prepare the cards, a card 
reader is used to read the data into the computer or onto tape, a card 
punch can be used to punch cards containing the record of the com- 
puter’s results, and a tabulator can produce a printed record from these 
cards. 

Punched cards are binary devices. Each discrete position on the card 
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Vig. 2-3. Coding for punched cards. (This figure was used on p. 41 of “Digital 
Computing Systems” by Samuel B, Williams, McGraw-Hill Book Company, Inc., 
New York, 1059, and on p, 25 of ‘High-speed Data Processing” by C. C. Gotlieb 
and J, N. P. Hume, MeGraw-Hill Book Company, Inc., New York, 1958.) 











22 Programming 


is a potential hole that is either punched or not punched. On a standard 
IBM card the positions are arranged in 80 columns and 12 rows. 
Normally a column contains the punched code for one numeral, letter, 
or symbol, as shown in Fig. 2-3. The particular codes were chosen for 
mechanical convenience in building accounting machines and as a matter 
of fact are relatively awkward for computers, but the computers can be 
programmed to handle any code, whereas the card machines cannot. 
When a binary computer produces cards that are to be used on the 
accounting machines, it must use the standard punched codes, but when 
the cards are to be used only for storing information that will later be 
put back into the computer, a more compact binary code can be used. 
Much more information can be packed into each column than just one 


24 | 25 


Y ' Li 
be? im? pt 

$1 Colon 00000 
Sar aap 71 9 pw rl 7 


4 iba feamaaiag 











an) 

2 (82 |2 212221222 
| ' 

3313.3.313.33 





1 t 
444444 44 
\ 14 
% (5519551555) 
Ni} ot o4 
HF |6 6i6 6 616 6 6 
S ieee 








704 COLUMN BINARY CARD 





4 |vn777777 
i] i] 

8 018 8 818 88 
i] ! 

9 9,999,998 


“Tin i 5 i 5 To OTT 3 
Tom LN3OT FORM I2I-N-2 




















Fig. 2-4. IBM binary card. Each of the sets of three columns, numbered across 
the top of the card, contains a bit-by-bit replica of the binary content of one 36-bit 
register in the computer (the IBM 704, in this case). The holes represent 1s and the 
nonholes Os. 


char acter. Figure 2-4 shows a card punched with a compact binary 
representation of information from the computer. 

Punched paper tape is used in the same way as punched cards, except 
that a deck of many cards is replaced by a continuous reel of tape. 
Paper tape usually has seven channels, as shown in Fig. 2-5, but tape 
with five, six, or eight channels is sometimes encountered. Paper tape is 
prepared on a special typewriter with an attached punch. A mechanical 
or photoelectric reader transfers the information from the tape into the 
computer. When the computer’s calculations have been done, a punch 
connected to the computer produces an output tape from which the 
information is printed by another special typewriter or equivalent 
device. 

Magnetic tape is very much like punched paper tape, with magnetized 
spots in place of punched holes, This use of magnetic tape is digital, 





Functional description of digital computers 23 


quite different from its use in recording the analog waveforms of speech 
or music. On digital tape, discrete spots are either magnetized or not. 
The degree or character of the 
magnetization is not relevant; there 
is only 1 bit of information per 
spot. Magnetic tape can be manip- 
ulated and stored more easily than 
paper tape or cards and can be 
read and recorded much more 
swiftly. It also has the advantage 
that it can be erased and reused. 
In some cases magnetic tape is 
prepared directly from a keyboard, 
but it may also be prepared from 
punched cards by means of a special 
card-to-tape machine. Magnetic- 
tape outputs can be converted di- 
rectly to printed records by spe- 
cially adjusted tabulators. 

Many computers have an on-line 
printer so that printed records can 
be made directly. This machine 
typically prints a complete line of 
60 or more characters at one time 
and operates at a maximum rate of 
150 to 1,000 lines per minute. Such 
speeds are impressive but are slow 
by computer standards; so on-line 
printers are used only when the 
amount of output is small. Other- 
wise the output is recorded on 
tape and is printed by an off-line 
machine. Fig. 2-5. Punched paper tape. (From 

Klectric typewriters are common __ T. C. Bartee, ‘Digital Computer Fun- 
on small machines and are some- 4@mentals,”” McGraw-Hill Book Com- 
mes found on lange, high-speed pany fae, New York 1860) The snall 
machines. An operator can type the sprocket wheels that move the tape 
messages and numbers into the com- in the machine. Each horizontal line 
puter from the typewriter keyboard, contains seven binary digits that en- 
and the computer can actuate the ode @ character. The binary code is 
arbitrary, depending on the mechanical 
design of the equipment. 
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typing mechanism automatically to 
report its results. By using a two- 
color ribbon, the typed record can show the operator’s input in one 
color and the computer's output in another, A typewriter is currently 
the best mechanism for man-computer conversations, 
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Some computers can produce a display on a cathode-ray tube (CRT), 
like the picture tube in a TV set. Moreover, the display can be photo- 
graphed by means of an automatic camera under the control of the 
computer. The computer can be programmed to draw graphs, complete 
with labels and legends, and to photograph these displays. Of course 
the CRT is not limited to graphs—any picture that can be formed by 
a series of spots can be produced if the necessary program is written. 
A picture can also be put into the computer by an optical scanner that 
changes the picture into a string of digits. 

Sometimes data exist in analog form. The data might, for example, 
be voltage waveforms recorded on magnetic tape. This tape cannot be 
read directly into a digital computer, since it does not contain sets of 
digits. However, a device called an analog-to-digital converter can be used 
to change the voltages into digital numbers proportional to the voltage 
amplitudes, which can then be stored in the computer. Since the voltage 
is a continuous function of time, the converter must also digitalize the 
time dimension. That is, it must convert the voltage amplitudes at fixed 
time intervals—say, every 0.01 sec. This sampling interval must be 
chosen appropriately for each particular problem. If the interval between 
samples is too long, some of the essential characteristics of the data may 
be lost, but if the interval is too short, successive amplitudes will be 
virtually identical and valuable computer storage space will be wasted. 

It is not possible to catalog further the variety of input and output 
mechanisms. As new problems arise, new machines are built to solve 
them. Direct read-in from a printed or typed page is possible by using 
a photoelectric scanner, and some commercial machines are already on 
the market, but the process is still largely in an experimental stage. At 
MIT and elsewhere, computer programs are being developed to allow a 
microphone to be used as a computer input device, and other equipment 
is being perfected to give a computer a mechanical voice. The user and 
the computer could then converse—literally. Presumably in the future 
even more amazing devices will be built. 


2-2 Storage 


The storage section, or memory, of a computer contains a large number 
of cells, or registers, each with a fixed capacity for storing information. 
Generally each register contains a single number, and the register 
capacity refers to the maximum number of digits in the number or the 
accuracy with which a single number can be represented. Of course it is 
always possible to achieve any desired accuracy by using more than one 
register to represent one number, but this is inconvenient. It is likewise 
inconvenient but sometimes economical to put two numbers together in 
a single register, All the information to be processed by the computer 
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must be located in storage registers. Additional registers are needed to 
store the results of intermediate calculations as well as the final results. 
The program itself, in a suitable digital code, must also be placed in 
memory. In all, a great many registers are needed—most computers 
have at least 1,000 storage registers, and some have as many as 65,000. 
In addition many computers have auxiliary memory devices in case the 
capacity of the main memory is exceeded. 

Storage registers can be thought of as pigeonholes, each holding an 
item of information for later reference. Perhaps a better analogy is a 
blackboard permanently lined to form a large number of squares, with 
each datum in a different square. At various stages of the program’s 
operation, the contents of a square may be copied to another square or 
processed and discarded. Some new datum or the result of some caleu- 
lation could then be placed in that square, after the old contents have 
been erased. 

To keep track of the stored data, each register is permanently labeled 
with a unique number, called its address. Access to a particular stored 
datum is obtained by specifying the address of the register in which it is 
stored. The addresses are assigned by numbering the registers se- 
quentially. The address of a register specifies the physical location of the 
register in the computer memory. Because a register is known by its 
address, and hence by its location, the term location is often used as an 
equivalent of the term register. 

The speed of computer operation depends ultimately on the time re- 
quired to obtain a number from a storage register. Data processing in 
the computer consists mainly in getting information from storage, com- 
bining it in some way with other data also obtained from storage, and 
then putting the results back into storage. Nearly every instruction in 
the program specifies the address of some register, the content of which 
is involved in the execution of the instruction. The program is also in the 
storage unit, and each instruction must be retrieved from storage before 
it can be executed. In short, the computer spends most of its time 
shuffling information into and out of storage. 

When the computer addresses a particular storage register, the 
clectronic circuits in the storage unit use the address to locate the 
required register and then to transmit the information. The time re- 
quired to locate and transmit the information is known as the memory 
access time. 

The fastest storage mechanisms in current use are magnetic cores, 
with a memory access time of 2 to 10 microseconds.! A magnetic core is 
a tiny, doughnut-shaped piece of ferrite that can be magnetized in either 
one of two polarities. Mach core can store 1 bit of information, so that a 


One microsecond » 0,000001 second, 
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set of cores is needed for a computer memory. Some large computers 
contain over 1 million magnetic cores. 

Medium-speed computers generally store information on the magnetic 
surface of a rotating cylinder called a magnetic drum. The storage regis- 
ters are rows of magnetized spots on the drum surface, access to which is 
achieved by a set of fixed reading and writing heads placed alongside the 
drum. As the drum rotates at a constant speed about its axis, the suc- 
cessive registers pass under the two sets of heads. Information can be 
stored in a particular register by activating the writing heads at the 
moment that register passes by. The information that is thus recorded 
can later be sensed by the reading heads. It is apparent that the reading 
and writing processes must be carefully synchronized with the drum 
movement. It is also clear that the length of time to gain access to any 
particular storage register on the drum depends on the position of the 
drum relative to the heads at the time the computer wants the infor- 
mation in that storage register. Up to a full revolution of the drum may 
be required before the storage register is positioned under the reading 
head. On the average, the memory access time is 2 to 10 milliseconds.' 

Besides speed, there is another important difference between drums 
and cores. It has already been noted that memory access time for a drum 
is not constant but depends on the position of the register in question, 
relative to the reading and writing heads. It turns out that this position 
depends on the last register previously addressed, because the processing 
unit is much speedier than the drum, requiring only a few microseconds 
per operation. When a word has been transferred for an operation, the 
processing unit is almost immediately ready for another word and the 
drum has rotated only a small fraction of a revolution. Hence the drum’s 
position when a particular register is being addressed is determined 
mainly by the location of the last register that was addressed. The 
memory access time is thus dependent on the sequence in which registers 
are addressed. On the other hand, the access time for every register in 
core is the same; it does not depend on any previous state of the machine 
and in particular does not depend on the sequence of addresses. Because 
any core register chosen at random is as accessible as any other, cores 
are called random-access memories. Drums are called nonrandom-access 
memories, because a register chosen at random is not in general as 
accessible as many others. 

When vast amounts of storage are required, computers generally make 
use of magnetic tape. Tape has the advantage of being able to store as 
many data as necessary, because more tape can always be added. Of 
course, since the data are located in one long string on the tape, the time 
required to gain access to any. particular datum depends on the speed 


1 One millisecond 0,001 second, 
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with which the tape can be moved past a reading head and may be of 
the order of several seconds or, with very long tapes, several minutes. 
Tape is definitely a nonrandom-access memory device. 

Large high-speed machines often have two or three different types of 
storage. Core storage is used for the main program and for the data cur- 
rently being operated on. Auxiliary programs are often stored on a mag- 
netic drum, and the majority of the data is often kept on magnetic 
tape. When an auxiliary program is needed, it is automatically read asa 
unit from the drum, into a block of core storage registers, whereupon the 
program can be executed at core speeds. Reading consecutive registers 
from the drum into the core can be done in a single drum revolution; so 
very little time is spent transferring the block. Data from the tape are 
also read as a block, and results are often stored as blocks on another 
tape. Again the transfer of blocks of computer words between tape and 
core storage minimizes the transfer time. All the computations involving 
the data are done when the data are in core, to take advantage of the 
high-speed random-access storage. 

The use of several types of storage on one computer is largely a matter 
of economy. Obviously, programming would be easier if all the routines 
and all the data were in a single random-access memory. However, core 
storage is expensive; many more registers of drum storage can be 
obtained for the same price, and tape is even cheaper. 


2-3 Processing unit 


‘The processing unit! of the central computer contains the circuitry for 
performing a variety of operations on the data. Standard arithmetic 
operations such as addition, subtraction, multiplication, and division 
are included, as well as many specialized operations. The unit also has 
one or more special registers for accumulating the results of the opera- 
tions. The main register, called the accumulator, is akin to the main set 
of dials on the carriage of a desk calculator. The accumulator usually 
has the same capacity as a storage register. Since the results of a multi- 
plication will in general be a number larger than the capacity of any 
single storage register, the accumulator usually has another register 
attached to it, forming one double register. The second register is called 
the multiplier-quotient register (MQ), or the Q register, or the B register, 
or some similar name. One machine uses the terms upper accumulator 
and lower accumulator. For some operations the two registers function 
independently, but for other operations, such as multiplication and 
division, the two function as one unit. As with the desk calculator, it is 
necessary for the programmer to know where in the double register his 


''This unit is often called the arithmetic (a-rith-met/-ic) unit, 
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result will appear. In division one of the registers contains the quotient 
and the other contains the remainder, whereas in multiplication one 
contains the most significant part of the answer and the other the least 
significant part. The specific details vary from one computer to the next. 

In addition to the arithmetic operations, the processing unit can also 
manipulate coded nonnumerical data, transfer data from one register to 
another, and perform other control functions. Most digital computers 
can perform at least 20 and sometimes more than 100 different oper- 
ations. A computer with only a small number of operations can ac- 
complish all the calculations that a more versatile machine can perform, 
but a larger number of alternative operations makes for greater efficiency, 
speed, and ease of use. The types of operations found on most computers 
will be introduced later, in the discussion of programming. 


2-4. Control unit 


A digital computer functions by interpreting and executing a sequence 
of instructions, each of which is a command for an operation to be per- 
formed. If a storage register is involved in the operation, then its 
address is included in the command. For example, the instruction ADD 
CONTENT OF 201 means “add the number in register 201 to the 
number in the accumulator.” Each instruction is coded numerically and 
the complete sequence of instructions, called a program, is placed in the 
storage unit at the start of the computation, along with the data for 
which the program has been prepared. The control unit coordinates the 
other units of the computer in order to execute the stored program. In 
doing this, it shuttles information back and forth between the processing 
unit and the storage element, as required, and also gets the successive 
instructions for itself out of the storage unit. 

The control unit operates in a two-step cycle. First, it gets the next 
instruction from storage and puts it in a special register in the control 
unit. In the second step, it decodes the instruction and sends the 
appropriate signal to the other units so that the instruction will be 
carried out. Very often the second step involves getting a data word from 
storage or putting a result into storage. Typically, then, there is a 
reference to memory in both steps of the cycle: one to get the instruction, 
and the other to manipulate the data. 

- One of the functions of the control unit is to keep track of the location 
of the program in storage. This is done by means of a special register 
called the program counter, or program location register. At the start of 


each control cycle, the program counter indicates the address of the’ 


storage register containing the next instruction in the program. When 
this instruction has been brought into the control unit, the program 
counter is advanced to the next program instruction, Normally, the 
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instructions are stored sequentially; so advancing the program counter 
means adding 1 to its present content. In other cases, the address of the 
next instruction will be contained explicitly in part of the current in- 
struction. In all cases each instruction includes the address of the next 
instruction in the program, either explicitly or implicitly in terms of 
some normal sequence. 

The control unit may also contain some special counters called index 
registers, which are used in connection with repetitive programs. As a 
general rule, some sections of a program are to be repeated a fixed 
number of times, perhaps with some slight modifications on each repe- 
tition. The index registers serve both to count the number of repetitions 
and to make the modifications. 

T he control unit is the heart of the computer. To understand how its 
various parts function, a basic knowledge of computer programming is 
needed. The description of programming in the next chapter will include 
a description of the functions of the various control registers and the 
dynamics of the control cycle. 
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Basic programming 


COMPUTER responds to instructions like an obedient clerk. Each 
A instruction commands the computer to select one of its built-in 
operations and to perform that operation on one of the items of infor- 
mation in the computer. Since the repertoire of built-in operations 1s 
quite elementary, a long sequence of instructions must be given for the 
computer to calculate anything interesting. A complete sequence of 
instructions is called a routine, or program, and the task of the pro- 
grammer is to invent a suitable sequence for the problem at hand. The 
elementary nature of the machine forces the programmer to specify the 
calculation procedure in minute detail, and the automatic nature of the 
machine demands complete accuracy, since the machine does just 
exactly what it is instructed to do. 

This chapter presents the basic concepts of programming and at the 
same time illustrates the dynamics of computer functions. As far as 
possible, the discussion is general, but in order to provide specific eX- 
amples, a hypothetical single-address binary high-speed computer will 
be described. Instructions will be invented to fit our convenience. The 
instructions conform to no actual computer; if the reader has a particular 
computer in mind, he may wish to test his own understanding of the 
basic ideas of this chapter by translating all illustrative examples into 
the particular instructions workable on his machine. 

The initial discussion of programming will completely omit any 
reference to input and output programming. We shall start by assuming 
that both the data for the problem and the program itself are already in 
the storage unit, and we shall carry our program no further than obtain- 
30 
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ing the required results in other storage registers. The process of getting 
the data into storage from cards or tape and the process of getting the 
results out of the computer require additional programs that are tedious, 
complicated, and specific to particular computers. It will be much 
easier to postpone to the next chapter all discussion of the input and 
output processes and to concern ourselves for the present with ma- 
nipulating the data inside the computer. 


3-1 Elementary programs 


As a start, let us write a program to perform the calculation 44 + 35 
— 19 = 60. The data 44, 35, and 19 must be located in specified storage 
registers. Any registers would do, but we arbitrarily specify registers 
200, 201, and 202. Likewise we specify that the answer shall be stored in 
register 220 after it is computed and that the program shall start in 
register 100. The program is shown in Table 3-1. 


Table 3-1 Program to compute 44 + 35 — 19 





Location E 
in storage Content of register Comments 
Program 
100 LOAD FROM 200 Get first number 
101 ADD CONTENT OF 201 Add second 
102 SUBTRACT CONTENT OF 202 Subtract third 
103 STORE IN 220 Record the result 
104 HALT 
Data 
200 +44 
201 +35 
202 +19 
Result 
220 — Answer 





To perform this program on the computer, the instructions and data 
must first be put into the specified registers with the proper digital codes, 
and the program counter in the control unit must be set to take the first 
instruction from register 100. Let us assume that this has been done by 
some input routine, and let us trace the results of each step in the 
computer operation as the program in Table 3-1 is executed. 

ach step is made up of two parts, lirst the computer examines the 
register named in the program counter to get the instruction to be 
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performed. Then the computer executes the instruction and at the same 
time adds 1 to the program counter. Table 3-2 shows this action in 
detail for our program. The contents of the accumulator and relevant 
storage registers are shown after the execution of each instruction. 
When the START button is pushed, the first part of the first step 
is carried out (step la): the control unit obtains its first instruction 
from the storage register specified by the program counter. In the 
example, the program counter started at 100; so the instruction ob- 
tained is LOAD FROM 200. In the second part of the first step (step 1b), 
the control unit executes the instruction by copying the number from 
storage register 200 into the accumulator, erasing anything that may 
have been in the accumulator. At the end of the first step the ac- 
cumulator contains the number 44. Also, in the second part of the first 
step, the program counter is advanced to 101. Instructions are stored 
sequentially in our hypothetical computer, and, unless directed other- 
wise, the computer always gets its next instruction from the next suc- 
cessive storage register. In the second step, the control cycle is repeated; 
the instruction ADD CONTENT OF 201 is obtained from register 101 
as specified by the program counter, and the instruction is executed by 
adding the content of register 201 to the content of the accumulator. 
Note that the number 201 in the instruction ADD CONTENT OF 201 
is not itself the number to be added but is the address of the storage 
register containing the required number. At the same time that the 
addition is being done, the program counter is advanced, so that at the 
end of step 2 the program counter contains 102 and the accumulator 
contains 79. In the third step the computer executes SUBTRACT 
CONTENT OF 202 by subtracting the number in register 202 from the 
number in the accumulator. At this point the accumulator contains the 
desired result. The fourth instruction, STORE IN 220, places the re- 
sult into the required register in storage. Then the computer halts. 

The example illustrates a rather general characteristic of digital- 
computer operation: when information is transferred to a new location, 
it is not erased from its old location. After the store instruction has been 
executed, the accumulator still contains the result. The process of storing 
a result from the accumulator into a storage register does not alter the 
accumulator. It does, however, erase the previous content of the storage 
register. Storing can be thought of as a copying operation. Likewise, the 
instruction LOAD FROM 200 does not alter the number in register 200; 
it puts a copy of that number into the accumulator, erasing whatever 
the accumulator previously contained. 

The cycle of computer operation provides an operational distinction 
between instructions and data. With both instructions and data occupy- 
ing storage registers, it might seem that some special numerical code 
would be necessary to specify which registers contain data, But the dis- 
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tinction is inherent in the two references to memory made on each cycle, 
one for the instruction specified by the program counter and one for the 
datum specified in the instruction. During the first part of the control 
cycle, the register addressed by the program counter is treated as if it 
contained an instruction; during the second part, the register addressed 
by the instruction is treated as if it contained data. 


Illustrative example: polynomial approximations. Many mathemati- 
cal problems include transcendental functions such as sines and cosines, loga- 
rithms and exponentials. A mathematician looks up values of these functions 
in printed tables when doing calculations by hand. With a digital computer, 
however, such problems are not handled by storing a table of each of these 
functions. It is more efficient to have the computer calculate a tabled value 
when the need arises. The computation is done by means of a polynomial 
expansion, essentially the same procedure being employed that was used to cal- 
culate the mathematical tables in the first place. Almost any function f(x) can 
be expressed as the sum of an infinite power series, 


f(a) = do + ax + ax? + age® + ayet + +s 


The values of the coefficients ao, a1, etc., depend on the particular function. 
The properties of these series may be found in any college calculus text. Here it 
will be sufficient to note that, by suitably choosing units and scale factors, 
series can be found such that each successive term is smaller than the previous, 
so that a few terms will give a good approximation to the function, and the 
approximation can be made as good as one pleases by using enough terms in the 
calculation. (For a detailed discussion, see Hastings, 1955.) 

A program to perform the calculation of the above function illustrates the 
frequently encountered necessity to rephrase the problem in order to use the 
computer efficiently. Suppose that five terms are sufficient. Then the equation 
can be rewritten as 


f(a) = ao + xf{ar + 2[a2 + (a3 + aux) ]} 


The program in Table 3-3 follows directly, going from right to left in the ex- 
pression, where we symbolize the actual values of the coefficients and the var- 
iables. The reader can verify that a program to calculate f(z) from the first 
form of the polynomial would be much less efficient. 


3-2 Contingent programs 


To show how computers make choices, and, incidentally, to give added 
significance to the program counter, we shall extend the problem of 
calculating 44 +35 — 19. Suppose that we now wish to calculate 
X + Y — Z for any three numbers X, Y, and Z stored in registers 200, 
201, and 202, Suppose, further, that we wish to place an arbitrary upper 
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Table 3-3 Polynomial approximation 





Location | Content of register Comments 
Program 
100 LOAD FROM 200 | as 
101 MULTIPLY BY CONTENT OF 300] az 
102 ADD CONTENT OF 201 | asx + as 
103 MULTIPLY BY CONTENT OF 300} asx? + age 
104 ADD CONTENT OF 202 | asx? + azu + ae 
105 MULTIPLY BY CONTENT OF 300 | aux? + asx? + aor 
106 ADD CONTENT OF 203 | asx? + asx? + ax + ay 
mie MULTIPLY BY CONTENT OF 300 | asx? + aga? + agv? + aye 
10 ADD CONTENT OF 204 | asx 3 2 
109 STORE IN 400 meee eee ee 
Constants 
200 a Coefficients 
201 a3 
202 a2 
203 a 
204 ao 
Data 
300 x Variable 
Result 
400 —_ f(x) 





limit of 75 on the answer to our previous problem. That is, register 220 
is to contain the result, unless the result is greater than 75, in which 
case register 220 is to contain 75. The program is shown in Table 3-4 

and its operation is traced in Table 3-5. The trace shows the content of 
the program counter at the beginning of each step in the computer 
cyclical operation, the instruction performed at that step, and the 
content of the accumulator and relevant storage registers after the 
instruction has been carried out. The first four instructions are the same 
its those in Table 3-1. But after the result is stored, it must be examined 
for size. Since the accumulator is not cleared during the storing operation 

it still contains the result after step 4. The program includes a cell, 
register 109, that contains the constant value 75; so 75 can be abe 
tracted from the result in the accumulator by executing the instruction 
SUBTRACT CONTENT OF 109. At this point, the number in the 
accumulator would be negative if the original result were less than 75 

while it would be positive if the original result were greater than or enial 
to 75, (Our hypothetical computer considers zero to be positive.) The 
choice is made by using a conditional jump instruction, The instruction 
JUMP IF MINUS 'TO 108 means that, if the accumulator contains a 
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Table 3-4 Example of program with conditional jump 
a 
Location 
in storage 


Content of register Comments 








Routine 
100 LOAD FROM 200 Get first number 
101 ADD CONTENT OF 201 Add second 
102 SUBTRACT CONTENT OF 202 Subtract third 
103 STORE IN 220 Result is obtained 
104 SUBTRACT CONTENT OF 109 Is it too big? 
105 JUMP IF MINUS TO 108 If no, skip to 108 
106 LOAD FROM 109 If yes, set result 
107 STORE IN 220 To upper limit 
108 JUMP TO 500 Go to next program 

Constants 
109 +75 Upper limit 

Data 
200 +44 
291 +35 
202 +19 

Results 

220 — Result 


I 


negative number, the normal sequence of instructions is altered and the 
computer skips to the instruction in register 108, omitting the in- 
structions in registers 106 and 107. This is accomplished simply by 
changing the program counter to 108 if the accumulator is negative when 
the instruction JUMP IF MINUS TO 108 is carried out. On the other 
hand, the program counter is advanced normally—to 106—if the ac- 
cumulator is positive when that instruction is carried out. Thus, if the 
result is greater than or equal to 75, the instructions in 106 and 107 are 
executed, with 75 left in register 203; otherwise register 203 contains 
the actual result obtained. Finally, the unconditional jump instruction 
JUMP TO 500 causes the program counter to be set to 500 uncon- 
ditionally, thereby taking the computer to another program that starts 
at register 500. 

The reader can easily see the difference that would result in the trace, 
shown in Table 3-5, if register 201 had contained +55 initially. In this 
case, the accumulator would be positive after cycle 5, and so the in- 
structions in registers 106 and 107 would be performed, 

‘There are always many different ways to program the same culcu- 
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7 ae trace shows the content of the program counter at the beginning of the step, the instruction performed at that step, the content of the 
accumulator after the instruction has been carried out, and the content of pertinent storage registers, where relevant, after the instruction has 


been carried out. 


37 








Basic programming 39 
38 Programming nas 
when the same calculation is to be done on a great many sets of data— 


lation. One very simple change would be to put —19 into register 202 triplets of numbers in our example—because the same short sequence of 
and change 102 to ADD CONTENT OF 202. This change is inconse- instructions with a few modifications can be made to serve for all sets. 
quential. A slight improvement in efficieacy can be made by replacing In our particular example, we would have either to shuffle the triplets, 
JUMP IF MINUS TO 108 by JUMP IF MINUS TO 500. A more one after the other, into registers 200, 201, and 202 or to put each triplet 
basic change would be to test the result ‘or size before storing. Indeed, in a different set of three storage registers and successively change the 
this would seem to be the natural thing to do. But the process of testing address parts of the instructions in registers 100, 101, and 102. The 
alters the result, and so we must record the result before testing it in latter is the normal process. Many computers have special instructions 
order to avoid the need for recalculating it. The reader can devise for to effect this address modification automatically. 
himself a routine that tests before storing. Consider the problem of adding a column of 40 numbers. Instead of 
To summarize, the fundamental operation of a digital computer 1s writing 40 add instructions each of which is executed once, we shall 
cyclical. On each cycle the computer obtains the instruction from the write one add instruction together with a few control instructions, all of 
storage register specified by the program counter, decodes and performs which get executed 40 times. The set_of repeated instructions is called 
this instruction, and advances the program counter. On each cycle there 2 loop. On each successive traversal of the loop, the address of the add 
are typically two references to storage: a reference to the storage instruction will be modified. We shall first write an add loop, using only 
register containing the instruction and then a reference to the storage the instructions that have already been introduced; then we shall 
register involved in the execution of the instruction. Conditional and introduce some new instructions. — 
unconditional jump instructions are used to regulate the sequence of The first program is shown in Table 3-6, and its trace is given in 
instructions executed, by altering the program counter. ‘Table 3-7. We assume that the numbers to be added are in registers 
Table 3-4 shows four different sets of storage registers. The in- 200 to 239 and that the result is to go in register 250, which is initially 
_-~ structions in the program occupy one set of registers (100 to 108), the set to zeTO by the instructions in 120 and 121. The other initial in- 
constants associated with the routine occupy the second set (109), the structions, in 122 to 124, will be discussed later. The loop is in 125 to 135. 
data for the program are stored in a thirc block of registers (200 to 202), The instructions in 125, 126, and 127 add the first number to the result. 
and a fourth block (220) is set aside for the results. Organizing the W hen they have been performed, register 250 contains 437. Then the 
program in this way helps the programmer to remember the functions instruction in register 126 is changed by adding 1 to it. To understand 
—~ of the various registers when errors must be sought or changes must be this, it will be necessary to reveal some of the details of coding in- 
made in the program. This organization is purely functional and does not st ructions in the computer. 
correspond with different sections in the computer. KF or the computer, An instruction consists of an operation, such as ADD CONTENT 
the routine is determined by the sequence of addresses in the program Ol’, and an address, such as 200. Both must be represented digitally in 
counter. If the address of a constant should get into the program counter order to be stored. Moreover, the digits must be combined so that the 
because of an error, such as an erroneous jump instruction, the computer instruction can be stored in a single storage register. The operation is 
would interpret that constant as an instruction and chaos would result. given an arbitrary digital code, while the address is a number; so its 
The sequence of instructions implicitly specifies the program’s organl- code is simply the digits comprising the number. Suppose that our 
zation for the computer. Any assignment of registers consistent with computer can perform fewer than 100 different operations, so that the 
the instructions is acceptable to the computer. Nevertheless, a neat operations can be assigned numerical codes in the range 00 to 99, and 
separation into routine, constants, data, and results is best for the suppose that ADD CONTENT OF is assigned the code number 46. 
programmer. Suppose further that there are fewer than 10,000 storage registers in the . 
computer, so that all storage addresses are in the range 0000 to 9999 ~) 
Bend und four decimal digits are needed to code the address. The complete 
3-3 Repetitive operations instruction is formed by placing the operation code before the address 
If we want no more than the simple eigh:-step calculation in the example code, so that the string of digits 460200 represents ADD CONTENT 
above, then, of course, pencil and paper has an advantage over a digital Ol 200. The computer is wired to interpret an instruction by taking the 


computer, Even if our sample program were an intermediate part of a first two digits as an operation code and the last four as an address. 
much larger calculation, the computer would not be earning its keep Now, the point of all this discussion is that the same digits can be in- 
during this part of the sequence, ‘The computer hecomes economical 
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Table 3-6 Program to add 40 numbers 


7 





Location Content of register | Pomernete i 
[eine ese eeeee re 
Program " 
“120 LOAD FROM 1 
121 STORE IN 250 Preset result 
122 STORE IN 151 Preset counter 
123 LOAD FROM ae Preset ADD 
STORE IN 
i LOAD FROM 250 Start of loop 
126 ADD CONTENT OF 200 | Add next number 
127 STORE IN 250 Save sum so far 
128 LOAD FROM 126 | Modify address 
129 ADD CONTENT OF 153 
130 STORE IN 126 a 
131 LOAD FROM 151 d 
132 ADD CONTENT OF a 
133 STORE IN Aa 
134 SUBTRACT CONTENT OF 154 Check if done 
135 JUMP IF MINUS TO 125 | Minus if not done 
136 JUMP TO 500 Go to next program 
Constants ih ies 
+ oO” Zero for initial preset 
i : For counting, changing 
i Lor Number of additions 
155 ADD CONTENT OF 200 Preset for 126 
Data 
200 +487 
201 +520 
239 +85 
Result 


250 — Sum 
460,200. Other numbers can be added 
1 can be added to 460,200, yielding 
rpreted as an instruction, means 
ess of an instruction can be 
rand adding or subtract- 


terpreted as a number, namely, 
to this number. In particular, 
460,201. This string of digits, when inte 
ADD CONTENT OF 201. Thus the addr 
changed by treating the instruction as & numbe 











Basic programming 41 
Table 3-7 Trace of program in Table 3-6 
Step Program Instruction Accumulator} Storaget 
counter ‘ 
1 120 LOAD FROM 152 0 
2 121 STORE IN 250 0 | 250 0 
3 122 STORE IN 151 0} 151 0 
4 123 LOAD FROM 155 | ADD CO 200 
5 124 STORE IN 126 | ADD CO 200} 126 ADD CO 200 
First 
cycle 
6 125 LOAD FROM 250 0 
7 126 ADD CONTENT OF — 200 + 437 
8 127 STORE IN 250 + 437 | 250 + 437 
9 128 LOAD FROM 126 | ADD CO 200 
10 129 ADD CONTENT OF 153 | ADD CO 201 
At 130 STORE IN 126 | ADD CO 201] 126 ADD CO 201 
12 131 LOAD FROM 151 0 
13 132 ADD CONTENT OF 153 + 1 
14 133 STORE IN 151 + 1] 151 + 1 
15 134 SUBTRACT CON- 
TENT OF 154 = 39 
16 135 JUMP IF MINUS TO 125 - 39 
Second 
cycle 
17 125 LOAD FROM 250 + 437 
18 126 ADD CONTENT OF 201 + 957 
19 127 STORE IN 250 + 957 | 250 + 957 
20 128 LOAD FROM 126 | ADD CO 201 
21 129 ADD CONTENT OF _ 153 | ADD CO 202 
22 130 STORE IN 126 | ADD CO 202}126 ADD CO 202 
23 131 LOAD FROM 151 + i. 
24 132 ADD CONTENT OF © 153 + 2 
25 133 STORE IN 151 + 2) 151 + 2 
26 134 SUBTRACT CON- 
TENT OF 154 = 38 
27 135 JUMP IF MINUS TO 125 - 38 
Fortieth 
cycle 
435 125 LOAD FROM 250 +34791 
436 126 ADD CONTENT OF — 239 +34876 
437 127 STORE IN 250 +34876 | 250 + 34876 
438 128 LOAD FROM 126 | ADD CO 239 
439 129 ADD CONTENT OF _ 153 | ADD CO 240 
440 130 STORE IN 126 | ADD CO 240} 126 ADD CO 240 
441 131 LOAD FROM 151 + 39 
442 132 ADD CONTENT OF 153 + 40 
443 133 STORE IN 151 + 40 | 151 + 40 
444 134 SUBTRACT CON- 
TENT OF 154 0 
445 135 JUMP IF MINUS TO 125 0 
446 136 JUMP TO 500 0 
AA7 500 





{| CONTENT OF is abbreviated CO in the Accumulator and Storage columns. 
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ing the appropriate amount. Also, the program can form an ee 
from its separate parts. If the instruction ADD CONTE) 
(460,000) is added to the rine 200 (000,200), the result is 
NT OF 200 (460,200). 
Pee ea, computer, numbers are assigned ree to 
the various operations, and the manner in which the numerical operation 
codes are combined with the address codes is also fixed by the designers. 
The codes differ from one machine to the next, but in all | Le 
digits in the stored instructions will represent the operation, er . S 
will represent the address, and instructions can be altered by treating 
mbers. 
gsr cei to our example in Table 3-6, we Pike to nar bee 
instruction in register 126 from ADD CONTENT OF 200 os a 
CONTENT OF 201, and so we must add 1 to the address. e = 
structions in 128, 129, and 130 accomplish this. The instruction e 
modified is placed in the accumulator, 1 is added to the accumulator, 
and the altered instruction is returned to its place in storage. ‘gas 
At this point our loop is nearly complete. We have set up A e = 
struction to form the required sum, step by step, by the instruc ee 
125, 126, and 127, and we have arranged to make the mpeenes modi : 
cations so that the next number in the list will be added to the sum o 
each successive cycle of the loop. We must now arrange for the aes 
to get out of the loop after 40 numbers have been added. The mon 
way is to count the number of times we have traversed the ae a 
when the count reaches 40. So we set up a counter, eae re Ae - 
we preset with zero at the start of the program, and we ' cas ? a 
counter on every cycle of the loop, by the instructions in 131, . a *3 
133. Then we test to see whether or not the count equals 40 by an sss : 
ing 40 from the accumulator, which is accomplished by the instruc io : 
in 134. If the count is less than 40, this subtraction will leave a is ‘ 
number in the accumulator. In this case, the loop must be pes ed; @ 
we JUMP IF MINUS TO 125. When the count reaches: 40, t ot : 
traction will leave 0 in the accumulator; so the conditional jump : a 
occur, and the loop is finished. The operation of the program is nen 
in Table 3-7; the first, second, and last operations of the loop is s - . 
If the reader understands the program Just presented, he unders pi 
the most important aspect of stored-program digital anaes oe 
ability to modify addresses by computation is the main source . i 
economy and flexibility of digital computers. The concept s apni 
simple, yet remarkably powerful. ae eT with s 
i ion, can do a great deal of work. pee 
ge eeratEe modification is the fact that there 1s no kote 
between data and instructions in storage. ‘I he content of register 1261 
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the example is treated both as an instruction and as a number. After the 
computer has executed the instruction in 125, it goes to 126 for its next 
instruction. It interprets the digits in 126 as a coded instruction and 
executes that instruction. Later, when the computer executes the in- 
structions in 128 and 120, it treats the digits in 126 as a number. 

The program illustrates another important characteristic of computer 
programs: the length and complexity of the program depend, not on 
the amount of work to be done, but on the complexity of the task. The 
same program would serve to add 4 numbers or 400 numbers—the only 
change required would be in register 154, which specifies how many 
numbers are to be added. Of course the time spent by the computer 
will depend on how often the add loop must be traversed, but the number 
of instructions in the loop is invariant. 

Presets. Loops are often part of larger loops. Thus a loop is often 
executed more than once in a program. In this case, it is necessary to 
ensure that the initial conditions of the loop are set when the loop is to 
be used again. All addresses that have been modified must be returned 
to their original value, and counters and temporary storage must be 
reset to 0. It might seem natural to clean up immediately after using a 
loop, to prepare for the next usage, but it turns out to be safer to preset 
the loop just before using it. In our example, the five instructions in 
120 to 124 preset the initial conditions. The register that will accumulate 
the results, 250, and the counter, 151, must initially be 0, while the 
instruction whose address gets modified must start as ADD 200. 


Illustrative example: table look-up. It is sometimes necessary to store 
u table of values of some function, say, F(X), for X = 0,1, 2,3,.... The values 
inight be stored in order in successive registers. Suppose that the table starts 
ut register 200; that is, register 200 contains F(0). Then 201 contains F(1), 
202 contains F(2), and so on. We need a program that will find the value F(X) 
vorresponding to a given X. 

‘The program is shown in Table 3-8. The value of X is stored in register 121, 
und the table starts at register 200. Since the value that we want, F(X), is 
stored in register 200 + X, our problem is to calculate the instruction LOAD 
"ROM (200 + X) and then to execute this instruction, winding up with F(X) 
in the accumulator. To calculate the required instruction, we add X, from 121, 
to a base instruction in 120, LOAD FROM 200. Then we put the calculated 
instruction into the next consecutive register, whereupon we execute it. 

Consider the concrete example of X = 5, so that 121 contains the number 5. 
‘The program starts by loading the instruction LOAD FROM 200 into the accu- 


mulator, The next instruction adds 5 to the accumulator, which now contains 
LOAD FROM 205. This instruction is stored in 103 by the instruction in 102. 
Vollowing the normal sequencing the instruction in 108 is executed after the 


instruction in 102, resulting in (5) being loaded into the accumulator, 
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Table 3-8 Table look-up 


Location Content of register Comments 
fo ee | 





Program 
100 LOAD FROM 120 Get start of table 
101 ADD CONTENT OF 121 Add argument 
102 STORE IN 103 Store table location 
103 LOAD FROM — Get value from table 
Constants 
120 LOAD FROM 200 Key to table 
121 x Argument 
Data 
200 F(0) Table 


201 F(1) 


. 


3-4 Index registers 


Address modification is such a frequent occurrence in computer programs 
that many computers include special components to simplify the process. 
These components are called index registers. An index register is a special 
kind of counter that both counts the number of loop cycles and modifies 
the necessary addresses by adding its count to a base address contained 
in storage. Before demonstrating how an index register works, let us 
first contemplate one of the many possible variations of the program 
just discussed to add 40 numbers: one that operates in very nearly the 
same way as a program with index registers. Almost every feature of the 
program in Table 3-5 has a possible variant. The addition can start at 
register 200 and proceed to 239, or it can be done in reverse order; the 
count can go from 0 to 40, 40 to 0, —40 to 0, etc. ; the count can occur 
before or after the addition in the loop; the add instruction can be 
modified in different ways. The program in Table 3-9 adds the numbers 
in reverse order, counts from 40 to 0, counts before adding, and modifies 
the add instruction by adding the count to a constant. First, the result 
is preset to zero, and the counter is preset to 40. In the loop, the counter 
is reduced by 1; if the result is not negative, the loop is not finished and 
so the count is added to ADD CC INTENT OF 200, yielding the required 
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Table 3-9 Variant of program in Table 3-6 





Location Content of register Comments 
Program 
120 LOAD FROM 152 
121 STORE IN 250 | Presets 
a — FROM 154 | Result = 0 
| panegees 151 | Counter = 40 
125 ST areacn cowain O ee ene 
2 Ru g F 153 | Reduce count by 1 
a ee IN 151 | Store count : 
ai secs IF MINUS TO 500 | Quit when count = —1 
D CONTENT OF 155 | Compute address: 
200 + count 
129 
ere aa 131 | Set add instruction 
250 | Add next number 
es ADD CONTENT OF — 
STORE IN 
+e eee “es Store result so far 
Constants 
151 — 
ae ; Counter 
= e Zero for initial preset 
7 er For counting 
Number of additions 
5 ADD CONTENT OF 200 
Data 
200-23 
9 Block of numbers 
Result 
250 a S 
Sum 





add instruction for this cycle of the loop. Then the addition occurs, and 
loc »P is repeated. Note that, on the first cycle, the counter will go fro 
0 to 39 and the first add instruction will be ADD CONTENT OF 239 
I'his is proper; the 40 numbers occupy registers 200 to 239. On the 1 t 
comy dlete cycle through the loop, the counter will go from 1 ‘4 0 a i 
add instruction will be ADD CONTENT OF 200. The next tim ie 
counter is reduced, it will become negative and he: loop will at : 
A program using index registers goes through the same steps ne j 
not using index registers, but some of the steps can be ‘oni ve 
oe of 8 hike instructions involving the index registers Seondes 
‘Ogiater acts like a counter, but it is a speci sti ‘ ei ar i 
control unit, not a storage register, The Bo aR Ai a Mi 
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be altered by special instructions and can be referred to by other 
instructions. In particular, address modification occurs automatically 
at the time that the modified instruction is executed. The instruction 
to be modified is coded in a special way so that the address that it 
specifies, called its effective address, is the base address plus the count in 
the index register. Let us use the notation 200:A to mean the base 
address 200 plus the number in index register A. Then, if index register 
A contains 39, the instruction ADD CONTENT OF 200:A is equivalent 
to ADD CONTENT OF 239. The base address 200 plus the index 
number 39 become the effective address 239. The effective address is 
formed in the control unit of the computer at the time that the in- 
struction is being executed, but it never gets stored in memory. The 
address in the indexed instruction remains unchanged—it is always the 
base address. 

The program for adding 40 numbers, using an index register, is shown 
in Table 3-10, and it is completely parallel to the program in Table 3-9. 
The presets are done in 120 and 121. Instead of presetting the result 
register to 0, we preset the accumulator to 0, because we shall keep the 
partial results in the accumulator until the addition has been completed. 
The counter is now index register A, which we preset by the new. in- 


Table 3-10 Adding 40 numbers with index registers 


Location Content of register Comments 
Pa ee 





Program 
120 LOAD FROM 152 Accumulator = 0 
121 LOAD A FROM 154 Preset index register A 
122 INDEX A FROM 153 Count down in A 
123 JUMP IF A MINUS TO 126 Quit when A = —1 
124 ADD CONTENT OF 200:A | Accumulate sum 
125 JUMP TO 122 Repeat loop 
126 STORE IN 250 
127 JUMP TO 500 To next program 
Constants 
152 0 0 for preset 
153 -1 For counting 
154 +40 Number of additions 
Data 
200-239 Block of numbers 
Result 
250 —— Sum 


Seca (nnn 
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struction LOAD A FROM 154, which puts +40, the content of register 
154, into A. The loop is in registers 122 to 125. Counting is done by th 
single new instruction INDEX A FROM 153, which causes the pie 
of register 153 to be added to the content of index register A, the result 
being left in A. The instruction in register 123 tests whether or not 
more numbers remain to be added. If so, the number in A will be vee 
tive ; if not, it will be negative. The actual addition is scobraplighed by 
the instruction ADD CONTENT OF 200:A, which has been explain d 
above. Then the loop is repeated. After all the numbers ate bed 
added, the loop exits to register 126, which instructs the re 
store the result. ee 
The difference between Tables 3-9 and 3-10 suggests the advant, 
m index registers. A more dramatic advantage occurs when the idee 
tape can be used to modify several addresses. The first example in 
ne chapter, 44 + 35 — 19 = 60, can be viewed as an instance of the 
aa ler X + Y — Z = R. Suppose that we have 100 X’s, 100 
: s, and 100 Z’s and want to find 100 R’s. Precisely, the problem is > 
0 compute Xi + Y; — Z; = Ri, from i = 0 toi = 99. The progra 
using index register A is shown in Table 3-11. It is only three Ganon 


Table 3-11 Program to compute X, + Y, — Z; = R; 


ce a a ee Be 





pocalen Content of register Comments 
Program 
100 LOAD A FROM 
! 108 = 
1s INDEX A FROM 109 he i 
JUMP IF A MINUS TO 120 
103 LOAD FROM 200:A : en 
as ADD CONTENT OF 300:A X, + Y; 
105 SUBTRACT CONTENT OF 400:A4 x, + y, —-Z 
106 STORE IN 500:A R; 
107 JUMP TO 101 Repeat loop 
Constants 
LO8 
+ 100 Preset for A 


109 ~ 
. For indexing 


Data 


200-299 
300-899 Xi 
400-499 7 
Results 
500-599 : 





ee 
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. ; di- 
initi i -1. Four instructions are mo 
tial program in Table 3 
nei as Sian e register. As an exercise the reader ar aan an 
wee Pacinnnilonis would be needed in the loop if index registers 
many 


used. 


i dias 
Ti tive example: N-way jump. Often in a program it is necessary 
ustrative Hy 


i i i the outcome of some 
j tial routines, depending on : 
eee a are aaa: example, suppose that a result se ae oe 
the result may be only 
i ister 120, and suppose that 
ae ei k aes 11.33, 4, and 5. Suppose that we want to Lea dion 
sear prea in 120 is 0 but that we want to jump to i ; 
i 


i i 240 if 4, and to 250 if 5. 
en ye os prc 3.12 will accomplish the required result. We set up a 
e prog 


previous calculation. 


Table 3-12 N-way jump 


i nts 
Content of register Comme 





Location 
acre Get number of alternative 
a pare oo on Jump to required jump 
i JUMP TO 200 Go if mee . : 
03 JUMP TO 210 Go if pete a 
i JUMP TO 220 Go if sna = 
05 JUMP TO 230 Go if ares a 
108 JUMP TO 240 Go if shales a 
107 JUMP TO 250 Go if numbe 
vi a — Number set by earlier 
program 


etic Wes Se ee 


g in 102 with JUMP TO 200. The aie 
ter 120, into index register A. Laeger 

i i uses the computer to jump to one of the instructions ak : Saale 
soriatiy 7 es the computer to jump to the required routine. : 3 7 
Wike a ae 120 is 2, index register A is loaded with the ea : eae 
Ue lanes of the instruction in Eni an een ee e Sie 
computer jumps to 104. The instruction in , 


to 220, as required. 


table of jump instructions startin: 
in 100 puts the result, from regis 


3-5 Subroutines 

i i i veral different 
Often the same sequence of instructions is needed at se 
places in a large program. Vor examp! 
of an angle is likely to be needed in many pat 


e, a sequence to compute the sine 
ts of a trigonometric calcu- 
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lation. Instead of writing the sequence every time it is needed, the pro- 
grammer can write the sequence only once, in the form of a subroutine. 
To do this, one or more special instructions are added, and the aug- 
mented sequence is placed in a section of memory apart from the main 
program. Whenever this sequence is needed, the main program jumps 
to the subroutine; when the subroutine is finished, the main program 
resumes. Used in this way, each subroutine is equivalent to a new 
computer instruction; extensive use of subroutines greatly increases the 
number and variety of instructions available to the programmer. 
Linkage. To make a sequence of instructions into a subroutine, 
some mechanism must be provided so that, at the conclusion of the sub- 
routine’s calculation, the main program will be resumed where it left off. 
l‘or example, suppose that we want to jump to a sine subroutine from 
location 125, after which the main program is to be resumed at register 
126. Later in the same program, the subroutine is to be entered from 
location 314, and the main program is to be resumed at 315. Still other 
entrance and exit points will occur further along in the main program. 
Clearly the last instruction in the subroutine must be a jump in- 
struction, and the address part of this instruction must be set in some 
way each time the subroutine is used, so that the computer will jump 
hack to the proper place in the main program each time. When the sine 
subroutine is entered from 125, its last instruction must be JUMP TO 
126; when entered from 314, its last instruction must be JUMP TO 315. 

There are many ways to preset the subroutine exit instruction and 
thus link the subroutine with the main program. Commercial computers 
use a variety of special techniques, most of which involve the program 
counter. Recall that the program counter keeps track of the address of 
the next instruction to be executed. Just before an instruction is exe- 
cuted, the program counter contains the address of that instruction : 
just after the instruction has been executed, the program counter con- 
(nins the address of the next instruction to be executed. Consequently, 
just before the program jumps to the subroutine, the program counter 
contains an address only 1 removed from the place to which the sub- 
routine is to return. The address in the program counter can be com- 
inunicated to the subroutine; it can be increased by 1 and stored as the 
uddress of the return point. 

‘The simplest mechanism is to put the content of the program counter 
into an index register. Thus we define a special subroutine instruction 
lor our hypothetical computer: SET A AND JUMP TO Y. When this 
instruction is executed, its own location is obtained from the program 
wounter and placed in index register A; then Y is placed in the program 
wounter, (SET BAND JUMP TO Y is similarly defined, as are parallel 
instructions for any other index registers.) Then the last instruction in 
the subroutine can be JUMP 'TO 1;:A, so that the main routine will be 
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Table 3-13 Subroutine linkage with index register 


Comments 


Location | Content of register 
pe aaa 





Main 
program 
(First part of main 
. program) 
5, into accumulator 
AD FROM 800 | Put angle in : 
ie aa A AND JUMP TO 500 | Enter subroutine; A = 125 
(Computer calculates sine, running through instructions 500-512) ; 
126 STORE IN 301 | Return from subroutine; 
. store sine 
(Main program continues) 
, into accumulator 
D FROM 802 | Put angle in 
aL a A AND JUMP TO 500 | Enter subroutine: A = 314 
(Computer calculates sine, running through instructions 500-512) : 
315 STORE IN 803 | Return from subroutine; 
. store sine 
: . (Rest of main program) 
Subroutine — a 
E IN 513 | First instruction in 
= 7 subroutine 


501 MULTIPLY BY CONTENT OF 514 


Subroutine calculations 


511 MULTIPLY BY CONTENT OF 513J 


‘ Subroutine exit; sine in 
TO 1:A | Su i 
sa aii accumulator 
513 — 
; Constants for subroutine 
517 —— 
Data ; — 
0 — First angle (calculated by 
~ main program) 
——_ Sine of first angle 
see Second angle (calculated 


by main program) 


803 —- Sine of second angle 


OR 
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resumed at the address A + 1. Table 3-13 shows how the subroutine 
instruction is used to execute a sine subroutine stored in registers 500 
to 517. The subroutine is entered at register 500, at which time the 
accumulator is to contain the magnitude of the angle. The final jump 
back to the main program is in 512, at which time the accumulator will 
contain the sine of the angle. 

Table 3-13 shows only the linkages. It is assumed that the main pro- 
gram finds the two angles as the result of some calculations and that the 
subroutine calculates the sine just as in the above illustrative example 
of polynomial approximations. Note that the subroutine communicates 
with the main program by way of the accumulator. When the sub- 
routine is entered, the angle is in the accumulator, and when the sub- 
routine returns to the main program, the sine of that angle is in the 
accumulator. It is apparent that, in essence, the instruction SET A 
AND JUMP TO 500 is equivalent to COMPUTE SINE. 

Calling sequences. When a subroutine has only one input, e.g., 
the angle in the sine subroutine, it can be delivered to the subroutine by 
way of the accumulator. The procedure for using the subroutine is to 
load the input into the accumulator and to enter the subroutine by 
executing a set A and jump instruction. There are many other ways for 
specifying subroutine inputs; for each way there is a corresponding set 
of rules for using the subroutine. The rules generally imply a certain 
sequence of instructions and/or constants in the main program and are 
known collectively as the calling sequence for the subroutine. 

A subroutine may have more than one input. A statistical subroutine 
{o compute the mean of a set of numbers might need to know how many 
numbers are in the set and where the numbers are located. The first few 
instructions in the subroutine would use its two inputs to modify itself 
for the particular problem. One way the main program can give the 
inputs to the subroutine is by putting them in successive registers in 
the main program immediately following the set A and jump instruction. 
If there are two inputs, they would be placed in two successive registers 
and the main program would be resumed in the third register after the 
set, A and jump instruction. The subroutine would obtain its first input, 
when needed, by executing LOAD FROM 1:4; the second input would 
he obtained by LOAD FROM 2:A. The subroutine would return to 
(he main program by JUMP TO 3:A. The calling sequence in the main 
program is illustrated in Table 3-14, where it is assumed that the sub- 
routine starts in 600. In this example, note that registers 246 and 247 do 
not contain instructions, even though they are in the main body of the 
program. The program encounters 245, which sends it to 600, and the 
subroutine returns the program to 248, bypassing registers 246 and 247. 
‘These registers are addressed by the subroutine and are treated as 
containing constants rather than instructions, 
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Table 3-14 Calling sequence with two constants 
Oe fe 


Location Content of register Comments 
Ne ee 








Main program 


(Main program) 


245 SET A AND JUMP TO 690 | Go to subroutine ; 
246 150 First input for subroutine 
247 300 Second input for subroutine 


248 STORE IN 500 | Return from subroutine 


(More main program) 
NN $$$ 80 
Illustrative example: bookkeeping. Consider a simplified bookkeeping 
procedure for the checking accounts at a bank. Suppose that a two-register 
record is kept in storage for each account, the first register containing the current 
balance in the account and the second the number of checks processed during 
the month. Assume that the code number for the account is the address of the 


first register of the pair. A subroutine is required to adjust the record of an 
aceount when a check is processed. The inputs to the subroutine are the code 


Table 3-15 Sample context for bookkeeping subroutine 


Location Content of register Comments 
eee ee 








Accounts 


1000 $ 120.57 1st account, balance 
3 1st account, 

ii number of checks 

1002 $4322.89 2d account, balance 
25 2d account, 

re number of checks 

1004 $ 560.14 3d account, balance 
10 3d account, 

ais number of checks 


Calling sequence ; 
SET A AND JUMP TO 300 Enter subroutine 


129 

130 1002 Address of account 
131 $ 175.00 Amount of check 
132 (Rest of main program) Return from 


subroutine 


————————————————————_ 
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number of the account and the amount of the check. The subroutine must 
decrease the balance by the amount of the check and must add 1 to the count 
of the number of checks processed. It must also test for overdrafts. A sample 
set of accounts is shown in the upper part of Table 3-15. The first three of these 
accounts have code numbers 1000, 1002, and 1004, respectively. 

Let us set up the calling sequence as follows: the subroutine shall be entered 
by the instruction SET A AND JUMP TO 300, the register following this 
instruction in the main program shall contain the address of the account, and 
the next register shall contain the amount of the check. A sample calling sequence 
starting in register 129 is shown in the lower part of Table 3-15. The actual 
subroutine is shown in Table 3-16, except for the part concerned with overdrafts, 
which is omitted. 

Let us follow the operation of the subroutine in response to the calling sequence 
in Table 3-15. When the subroutine is entered, index register A contains 129. 
The effective address of the first instruction is thus 130; so the instruction loads 
the number 1002 into the accumulator. Then the next two instructions cause 


Table 3-16 Subroutine to compute checking-account balance 


Location Content of register Comments 

Program 
300 LOAD FROM Get location of account 
301 STORE ADDRESS IN Set to load balance 
302 STORE ADDRESS IN Set to store balance 
303 ADD CONTENT OF +41; location of tally 
304 STORE ADDRESS IN Set to load check tally 
305 STORE ADDRESS IN Set to store check tally 
306 LOAD FROM Get balance 
307 SUBTRACT CONTENT OF Reduce by check value 
308 STORE IN Replace new balance 
309 JUMP IF MINUS TO Jump if overdraft 
310 LOAD FROM Get number of checks 
311 ADD CONTENT OF +1; tally 
312 STORE IN Replace new number of 

checks 

313 JUMP TO : Subroutine exit 

Constant 
314 +1 Constant 

Subroutine 

program 


B15 (Routine for overdraft) 
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this number to be stored in the address parts of 306 and 308, which now read 
LOAD FROM 1002 and STORE IN 1002, respectively. (The instruction 
STORE ADDRESS IN 310 concerns only the digits in a computer instruction 
that encode the address. These digits from the accumulator replace the corre- 
sponding digits in register 310.) Next the number in the accumulator is 
raised to 1003, and registers 310 and 312 are changed to LOAD FROM 1003 
and STORE IN 1003, respectively. Then the instruction in 306 is carried out, 
loading $4322.89 into the accumulator. The effective address 2:A is 131; so the 
balance is reduced by $175.00, and the result is stored back in the account- 
balance register. Since there is no overdraft, the computer goes on to 310, where 
the number of checks, 25, is put in the accumulator, increased by 1, and stored 
back in the proper account register. Then the subroutine returns to 3:A, or 132. 


Illustrative example: subroutine linkage without index registers. 
Subroutines can be linked to the main program by using only the simple load 
and store operations. The method is cumbersome and inefficient, but it works. 
Among the stored constants for the main program we shall include one jump 
instruction for every reference to the subroutine. Before entering the subroutine, 
we shall load the appropriate jump instruction and store it in the last instruction 
location of the subroutine. Suppose, as in the above example, that a sine sub- 
routine is stored in registers 500 to 517. The subroutine is to be entered at 500, 
at which time the accumulator is to contain the magnitude of the angle. The 
final unconditional jump back to the main program is in 512, at which time the 
accumulator will contain the sine of the angle. (Registers 513 to 517 contain 
constants.) The sequence of instructions in Table 3-17 will compute the sine 
of the angle whose magnitude is in register 800 and will store the result in 801. 
The instructions in 122 and 123 preset the subroutine exit instruction. Then 
the input to the subroutine is placed in the accumulator, and the subroutine is 
entered. When the subroutine reaches the instruction in register 512, which now 
reads JUMP TO 126, the main routine is resumed and the result is stored. 
The instructions in registers 122 to 125 are the calling sequence for the sub- 
routine. 


3-6 Some computer idiosyncrasies 


All computers operate according to the principles described in this 
chapter, but there are some procedural differences worth noting here. 
Of course, the particular set of available built-in operations will differ 
from one computer to the next, but there are also differences in program 
sequencing, indexing, and address modification. 

Almost all high-speed computers and many medium-speed computers 
store instructions sequentially, so that the computer gets its next in- 
struction from the next successive storage register unless otherwise 
directed by jump instructions. A few computers, including the IBM 650, 
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Table 3-17 Subroutine linkage with LOAD and STORE 


TT 


Location Content of register 





Comments 
Program 
: (Start main program) 
122 LOA 
_ ae pi oa jump instruction 
ro) reset subroutine exit 
LOAD FROM 800 Get angle 


125 JUMP TO 
; : 500 Enter subrouti 
(Subroutine calculates sine, running through instructions in 600-512) 


126 
STORE IN 801 Store result from 
. subroutine 
; (Rest of main program) 
Constant 


400 JU 
MP TO 126 For subroutine exit 


Subroutine 


500 ST 
ORE IN 513 First instruction in 


: subroutine 


501 MULTIPLY BY CONTENT OF 514 


Subroutine calculations 


511 MULTIPLY BY CONTENT 
512 JUMP TO re 


Subroutine exit; preset 
si by calling sequence 


Constants for subroutine 
517 


Data 


800 ——. 
x01 oe Magnitude of angle 


Sine of angle 


SS 


require that each instruction contain not only an operation and th 
address of the data to be operated on but also the address of ihe ae 
instruction to be performed. A sample instruction would b TaD 
!' ROM 201 AND GO TO 101. Such a computer does nis 

standard automatic sequence built in, but each instruction e licitly 
curries an address showing where the next instruction in the a a i 
to be found in storage, The program counter is advanced by Latipigns 
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the address specified in the instruction, rather than by adding 1. Such a 
design is useful in drum computers, for example, where the registers in 
storage are not equally accessible, and it may be to the programmer’s 
advantage to place his instructions in locations in the storage so that, 
when the time comes for them to be executed, they will be most readily 
accessible. 

Indexing procedures vary widely. Some computers do not have index 
registers but have some equivalent operation for repeating a loop. Others 
have no built-in procedure for repetition, so that addresses must be 
modified explicitly. Moreover, computers with index registers do not 
compute effective addresses in the same way. Some, such as the IBM 
704/709 and their relatives, subtract the number in the index register from 
the base address to obtain the effective address. Thus, if the base address 
is 200 and the index register contains +10, the effective address is 190, 
not 210. There is no strong advantage in either system. 

The explicit system of indexing—adding | to an instruction—requires 
care in application. On some computers, some instructions are encoded 
by digits that, when interpreted as a number, represent a negative 
number. If, for example, the digits encoding STORE IN 200 repre- 
sented a negative number, then | would have to be subtracted to obtain 
STORE IN 201. Normally the problem will not arise on computers 
that do not have index registers, but computers with index registers may 
have many “negative” instructions. These computers, including the 
IBM 704/709, were not designed for explicit address modification; the 
designers intended that address modification should always be done by 
index registers. 

Finally, the reader should be warned that, when he turns to the pro- 
gramming manual for some actual machine, he will find the operations 
listed more concisely than in this chapter. Prepositions and other de- 
scriptive phrases are omitted. For example, LOAD FROM is simply 
LOAD, ADD CONTENT OF is ADD, STORE IN is STORE, and 
JUMP TO is JUMP. Other names are often encountered for the same 
processes. JUMP is often called BRANCH, or TRANSFER. LOAD 
is sometimes called CLEAR AND ADD, meaning ‘‘Clear the accumu- 
lator to zero, and then add the contents of the addressed register to the 
accumulator.” The manual may also suggest an additional stage of 
abbreviation, such as LOD, ADD, STO, and JMP. The operations do 
the same things, but their names are different. The instruction ADD 
201 still means “Add the content of register 201 to the content of the 

accumulator.” The longer names with explicit. prepositions seem to 
clarify the procedures for beginners, whereas the abbreviated names are 
- more convenient when the programmer has the meanings of the oper- 
ations clearly in mind, 
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Problems! 


1. Write a program to perform each of the following calculations. In each case 
indicate what the trace of your program would show (see Table 3-5): 


a. 44+ 35+ 19 = Ans. b. 19 — 44 + 35 = Ans. 
c. 19 — 35 = Ans. d. 88 — 35 = Ans. 


2. For each of the following processes, write a program to produce the required 
result. ANS is a symbol meaning the computed answer. 


. Jump to 300 if X is greater than Y; otherwise j 
: se Jump to 400. 

. ANS = (X — Y) or X, whichever is larger. 
ANS = (X — Y) or (Y — X), whichever is positive. 
. ANS = (Z + X) or (Z + Y), whichever is larger. 
a = the largest of the three numbers X, Y, Z. 

ut the largest of the three numbers X, Y, Z th 
] X, Y, to 201, th x 
into 202, and the smallest into 203. seth 
g. Determine how many of the numbers X, Y, Z are negati i 
count (0, 1, 2, or 3) into 200. _ Bea eal 


yeanoese 


3. pen a loop is to be used very often in a calculation, it must be kept as 
short as possible. Shorten the loops in Tables 3-6, 3-9, 3-10, and 3-11 by at 
one instruction, by changing the order in which operations are per- 
formed and/or the manner in which the count is made. (The conditional 
instruction JUMP IF A PLUS TO Y may be used; see Appendix.) 

4. se Sa gene Aes branch can be done without index registers 

5. Show how the illustrative program for table look- % 
ee. able look-up can be improved 

6. Twenty numbers are stored in successi i i 

ive register : 
register of this block is 200. ° ences ae 


a, Write a program to add 5 to each numb ing it in i 

ite a er (repl s initi 
es (replacing it in its initial 
b. Write a program to make each n iti i 

umber positive with i 

magnitude of the number. ee 
Cs Write a program to rearrange the numbers so that they are in a descend- 

ing order of algebraic size (for example, 5, 2,0, —3,...). 


7. The responses of 100 examinees to a particular item on a five-alternative 
multiple-choice test are coded as 1, 2, 3, 4, or 5 and are stored in 100 suc- 
cessive storage registers, the first of which is 200. 


‘The | Pat : 

rhe Appendix lists the hypothetical computer instructions that may be used in 
programming the problems, Alternatively, the list of instructions available on som 
specific computer may be used, iN 
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10 


a. Write a program to tally the distribution of responses in a set of five 
registers, the first of which is 300. 

b. Using the result of (a), write a program to find which is the most popular 
response. 

c. The responses of these same examinees to a second item are similarly 
coded and stored in the same order in 100 registers, the first of which is 
400. The correct answers to the two items are 4 and 3, respectively. 
Count the number of examinees who answered both items correctly. 

d. Assuming the conditions specified in (¢), find the complete 2 X 2 table 
of right and wrong answers to the two items. 


. English text is coded letter by letter, with A=1,B=2,...,2= 26, and 


space = 0, with all punctuation ignored, and is stored in successive registers 
of storage, one coded symbol per register. Assume that 200 is the first 
register in a block of 500 coded symbols. 


a. Write a program to count the number of words (assume that the text 
starts at the beginning of a word and ends at either the end of a word 
or a space). 

b. Write a program to count the number of four-letter words. 

c. Write a program to count the number of double letters, ¢.g., the TT 
in Little, but not the two L’s. 

d. Write a program to recode each symbol according to the substitution 
specified by a list of 97 registers, the first holding the new code value 
for space, the second the new code value for A, ..., and the last the new 
code value for Z. 

e. Write a program to tally a distribution of word lengths (e.g., the number 
of words of length 1, 2, 3, 4,.--; with the tenth category for words of 10 
or more letters). Make a similar tally for pseudo words defined as the 
number of symbols between successive E’s. (Be sure to provide for the 
possibility of EE.) (See Miller and Newman, 1958.) 


. Write a subroutine to process a block of numbers, determining how many 


of the numbers are greater than a certain criterion value. The inputs to the 
subroutine are the criterion value, the location of the first register in the 
block, and the number of registers in the block. The calling sequence is to 
be as follows: The subroutine is to be entered by SET A AND JUMP TO 
500. When the subroutine is entered, the accumulator is to contain the 
criterion number, register 1:A (the next consecutive register after the Set A 
and jump to 500 instruction) is to contain the address of the first number in 
the block, and register 2:A is to contain the number of registers in the block. 
The subroutine is to return to 3:A with the required count in the accu- 
mulator. 

A general-purpose subroutine may have a variable number of input param- 
eters that the programmer wishes to specify in the main routine. How can 
the calling sequence be set up so that the subroutine will get all of its inputs 
and will return to the proper place in the main routine, no matter how many 
rogisters are used for the calling sequence? 


4, 


Programming techniques 


8 digital-computer programming has so far been presented, the 
job must appear dismal and exacting. It would seem shat the 
programmer must analyze and dissect his problem into its smallest 
components so that he can translate it into a string of elementary 
computer instructions. Although it is true that meticulous attention to 
detail is required, there are many techniques and shortcuts that help 
lessen the chore. This chapter will discuss symbolic coding, which re- 
moves the burden of address assignment from the hemtamisers flow 
charts, which help in planning complex programs; program iiieatics 
which permit the use of ready-made programs and subroutines; dail 
special “debugging” programs, which assist in locating errors in pro- 
grams. 


1-1 Symbolic coding 


In the programs presented in the preceding chapter, the instructions and 
the data are shown in specific registers. Consider the trouble that would 
arise if the data for these programs were moved to new storage locations 
or if it became necessary to add another instruction somewhere in the 
midst of the program sequence. Many of the addresses would have to be 
changed, and so the program would have to be completely rewritten 
lor this reason, programmers do not want to commit themselves to the 
use of particular absolute addresses but prefer to write the program in 
symbolic notation. Then, if an instruction has been inadvertently 
omitted, it can be inserted in the program in symbolic form without 
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disrupting the address sequence. The absolute addresses can then be 
specified after the program has been completed. 

Table 4-1 shows one possible symbolic version of the absolute program 
in Table 3-11. Addresses that the program refers to have been given 


Table 4-1 Program with symbolic addresses} 


eS UU E EEE EEE 








Location | Content of register Comments 
Program 
START LOAD A FROM N Preset index 
register A 
LOOP INDEX A FROM MONE Reduce A by 1 
JUMP IF A MINUS TO NEXT Tf done, go to next 
program 
LOAD FROM X:A Compute next set 


ADD CONTENT OF Y:A 
SUBTRACT CON- 


TENT OF Z:A R; = xX; + Y; = Z; 
STORE IN RESULT:A | Store answer 
JUMP TO LOOP Go to repeat for 
next set 
Constants 
N +100 Preset for A 
MONE - 1 For indexing A 
Data 
x Block of 100 registers 
Y Block of 100 registers 
Z Block of 100 registers 
Result 
RESULT Block of 100 registers 


a 


+ The program calculates X; + Yi — Zi = Ri. 


arbitrary symbols of the programmer’s choice—usually short words or 
abbreviations that serve as mnemonics as well as symbols. For example, 
the programmer has used MONE to name the cell holding minus one, 
(—1) thus reminding himself of the content of the cell. Addresses to 
which no reference is made need no symbols; they will be assumed to 
occupy sequential registers unless otherwise specified. 

_ Symbolic notation has so many advantages that it should always be 
used in programming problems of any complexity, There remains, how- 
ever, the additional task of translating from symbolic to absolute 
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notation. This task is completely routine and can be performed by any 
accurate clerk. It can, in fact, be done by the computer. Most computers 
have special symbolic assembly programs that will automatically make 
specific address assignments to all symbolic locations when given a few 
simple directions. Thus, in the exercise in Table 4-1, if we specify that 
START = 100, X = 200, and NEXT = 120 and that X, Y, Z, and 
RESULT name the first registers in consecutive blocks of 100 registers 


SAP CODING SHEET 





































































































Problem No,_4 ~/ Programmer BELT GREEN Date // Dec /96/ Poge __/ of _/ 
7 
LOCATION OPERATION ADDRESS ; COMMENTS 1.0, 
EETEA EP EEEEL POPE AERTE SAAT TOPE ESTEE TAP ESTT ESTE APE PETES TT SOS ESTER ST PEEPS ERTS 
START LDA _N, ‘ PRESET LL AL 4 ' : | 
L9GP__IXA MONE + + LEDUCE A BY I A as A 41 
7 JAM, NEAT . . LF-6$ 70 NEXT, PRdoRAM i re i 
£Od, XA ; - 4LPMPUTE NEXT SET, ri eh 
Aco, YA i = + = + = +t + \ 
SUB, ZA + + + += + + +— A 
5TP , LESULT? AL " ST@RE ANSWER ‘ 7 J re 
. IMP, LOBP : - 9 TO REPEAT NEXT SET 4 f ‘ 
N_ DEC, +100 _ ‘ FEESET FOL A ; > + 
MONE DEC =f; ; ; FRE INDEXING, A * m 7 
X__ BLK, 100 . ‘ pXiCE) 4 aN . 
_Y BLK, 109 ‘ ‘ Ycr) zy : . 
2 BLK, 100 : 2) ‘s ; : ' ral 
MOSULT | BLK, —/00 A ‘ -F 1)= (1) + CI) - 2G) = : 4 
+ + + + + + + —+————+ +- 
4 + + 4 + + + + +— + 
= + + + : + + + + pitiacin 
+ soe + + + + + + + + 
+ + + + + + . + 4 
~ + + + + + + + + + 
—_ + ——_+-— —+ + + + + + + . 
+ +—_+ + + + + + t + 4 
Te + a ea ——— + + + + 
—I— s+ + t + + + + 
L >= a. —1— = 








- n 4 n ——1.. io — 
VESSEC7TO901 234567 8901 2545678901 2345678901 23456TETOIZSSSSETOXIOIZS456TOIOI2SASETAIG 


Vig. 4-1. A symbolic coding form. The operation abbreviations can be deduced 
by comparison with Table 4-1. DEC indicates that the register is to contain the 
number whose decimal equivalent is in the Address column. Note the convention that 
the letter O has a slash drawn through it to distinguish it from the numeral 0. 


cach, the computer has enough information to produce the program as 
shown in Table 3-11. 

‘lo use the symbolic assembly program, the programmer prepares a 
coding sheet like Fig. 4-1. Each line on the sheet is punched in a card, 
according to a specified format. The card contains the ‘ocation symbol, 
if any; the symbolic instruction, which is composed of an abbreviation 
for the operation and a symbolic address; and any comments the 
programmer wishes to record, Simple rules must be followed; e.g., a 
standard, prescribed abbreviation must be used for each computer 
operation, the location symbol must have no more than six characters, 
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etc. Program constants and storage blocks are indicated by special 
symbols in the operation columns. These symbols, such as DEC _and 
BLK in Fig. 4-1, are not translated into operations but signa the as- 
sembly program that some special action must be taken. DEC, for ex- 
ample, signals the assembler the register is to contain the digital 
equi nt of the number in the address column, whereas BLK signals 
that a block of registers is to be set aside, the number of registers in that 
block being specified in the address columns and ‘the symbolic name of 
the~first-register in the block being specified in the location column. 
Separate cards -are-prepared for the location information, e.g., START 
= 100. The complete deck of cards is fed into the computer, where 
absolute addresses are assigned to all address symbols. Then each 
instruction is assembled from its parts. Thus, the first instruction in 
Table 4-1 is set to go into register 100, its abbreviated operation LDA is 
translated into the corresponding digital code, for example, 20000, and 
the address for N is combined with this, for example, 20108. The com- 
ments are ignored, but they are printed on a record, together with the 
symbolic form of the instruction and the machine’s digital translation, 
for the programmer’s use in checking the program. Such a record is 
called an assembly listing and is the main document used by the pro- 
grammer in testing and debugging his program. The output of the 
assembly program is the assembly listing and a new deck of cards con- 
taining a compact digital version of the assembled program. The 
assembled deck is put into the computer when the program is to be run. 

Symbolic assembly programs have some primitive error-catching 
abilities. If the programmer makes the mistake of using the same symbol 
for two different registers, the program will print an indication of the 
error on the assembly listing. If an address symbol has been used as part 
of an instruction but has never been put in the location part of some 
card, then the computer will not be able to make a translation and will 
print an indication of the error. The assembly listing provides an 
explicit record of such oversights, and it also shows the programmer what 
his program looks like when translated into the machine’s digits. A 
glance at this listing may be sufficient to sce whether or not the trans- 
lation looks reasonable; peculiar address assignments or odd-looking 
numbers may signify some undetected error. 

Symbolic assembly programs sometimes include provision for other 
special notation. For example the symbolic address LOOP +3 might 
be a permissible notation to refer to the third location after the register 
whose symbolic address is LOOP. Such special devices will not be used 
in this book, since they tend to be different for different computers and 
different symbolic assembly programs. However, symbolic notation will 
be used extensively, 
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4-2 Flow charts 


When computer programs are long, with many loops and contingencies, 
the logical structure can best be understood by means of a block diagram, 
or flow chart. Many programmers prefer to start a new program by 
planning its structure on a flow chart, specifying all the contingencies, 
and showing the path that the program will take following each possible 
outcome. When the relationships among all parts of the program have 
been carefully charted, the instruction-by-instruction programming can 
be done, fairly readily. 

Standard flow-chart symbols should be used, to help others who may 
read the chart (see Grems, 1960). A rectangle with one input and one 
output (Fig. 4-2a) indicates a block of processing. A comment is enclosed 


1 


———_—_—__1___» 


c = <Gee> 
No 
Yes No 
a 


Fig. 4-2. Flow-chart symbols. (a) Block of processing; (b) comment; (c) choice 
points; (d) input-output. 


in a rectangle that is attached by a dotted line to some flow line (Fig. 
1-2b). A choice point is indicated either by an oval or by a diamond 
(I°ig. 4-2c). Input and output are depicted according to the type of device 
used, e.g., card, tape, printed page (Fig. 4-2d). 

I'low charts are useful, not only in coding the program, but also in 
oxplaining the program to others. A knowledge of the general outline 
of the program may be helpful to those who want to use the program and 
Will be invaluable to anyone wanting to modify the program. Also, if 
someone wishes to make a copy of the program for a different computer, 
he can work more easily from a flow chart than from the program itself. 


Illustrative example: distribution of word lengths. Suppose that a 
passage of English text has been coded and punched on cards, with numerical 
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codes for the letters and space and with all punctuation ignored. A program is 
required to find the distribution of word lengths for words in the passage. As- 
sume that the passage has been subdivided into blocks of 100 words. The flow 
diagram for one possible program is shown in Fig. 4-3. A block of words is read 
in from cards and is processed letter by letter. Each letter is counted until a 


START 













Set letter count to zero 


Yes 


Count is 
—-+ limited 
to 20 


Finished with 100 words? 
Yes 















Add one to 
tally of words 
of length equal 
to letter count 





Add one 
to letter 
count 


Finished with passage? 
Yes 


Print out distribution 
of word lengths 


STOP 


Fig. 4-3. Flow diagram for tallying distribution of word lengths. 


space is reached or until the count reaches 20. The count is the length of the 
first word; and the tally of the number of words of that length is increased by 1. 
The letter count is reset to 0, and the next word is counted. The tally for the 
count of 20 is the number of words having 20 or more letters. The process con- 
tinues to the end of the block, As each block is finished, the next is read in, until 
the entire passage has been processed. ‘Then the distribution is printed out, and 
the program stops, 
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4-3 Program libraries 


Every computer installation maintains a library of routines and sub- 
routines that can be used as parts of other programs. Such libraries 
usually include subroutines to compute trigonometric functions, loga- 
rithms, exponentials, square roots, and random numbers. Occasionally, 
subroutines can be found for more specialized functions, such as Bessel 
functions, areas and ordinates of the normal curve, matrix inversion, 
and the like. In addition, the library is likely to contain a number of 
complete computer programs, which may either be used as they stand or 
be converted into subroutines. For example, complete programs for 
analysis of variance, matrix roots and vectors, and factor analysis are 
available at some installations. 

It is obviously a great advantage to find an existing computer program 
that will do what is wanted. Recognizing this, the users of each computer 
have made formal arrangements for sharing programs. The arrangements 
take the form of a users’ organization sponsored, at least in part, by the 
computer manufacturer, for whom the existence of such a group is a 
strong selling point for future customers. The users’ group establishes 
programming conventions to minimize confusion in exchanging pro- 
grams. New programs or program descriptions are submitted to the 
users’ group, which meets regularly and distributes this information to 
all members. The major users’ groups and the computers they represent 
are: SHARE—IBM_  704/709/7090; GUIDE—IBM 702/705; USE— 
Univac 1108A/1105; Univac Users’ Group—Univac I, Il; DUO— 
Datatron; EXCHANGE—Bendix G15; POOL—LPG 30. Information 
about these and other groups will be found in the Communications of the 
Association for Computing Machinery. The most prevalent computer— 
the IBM 650—has no formal users’ group, but information about avail- 
able 650 programs is often carried in the Communications, and informal 
rrangements can be made to exchange programs. 

liven if one finds an existing program to do the job, some small modifi- 
cations will generally seem necessary. If the program was written at a 
different computer center, the local ground rules may be different—in 
the use of certain input and output tapes, for example—and so a few 
changes must be made. Also, the program may not be clearly docu- 
mented, so that some trial and error is needed to learn how to use it— 
for example, how to get the input in the proper format. In short, there 
is always some preliminary work to be done even when an existing 
program is to be used, and some knowledge of programming is required 
for this work. 

The routines and subroutines in the program library are in symbolic 
form and generally exist as decks of cards that can be reproduced and 
inserted in the programmer’s own symbolic card deck, ready for the 
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symbolic assembly program. Subroutines that are used often are some- 
times stored on a library tape accompanying the symbolic assembly 
program, so that they can automatically be inserted into programs being 
assembled. The programmer merely prepares a special symbolic card, 
indicating that this subroutine is needed, e.g., SUBROUTINE SINE. 
This card acts as a surrogate for an actual deck of cards containing 
the subroutine and indicates to the symbolic assembly program that 
the sine subroutine is to be inserted from the library tape. A slight 
problem arises when a library subroutine is inserted either by the pro- 
grammer or by the assembly program, because the programmer who 
prepared the subroutine may have used some symbols that have also 
been used by the author of the program into which the subroutine is 
being inserted. A special procedure of some sort is provided to keep the 
two sets of symbols from clashing. The procedure depends on the par- 


'. ticular assembly program being used; here we merely direct the pro- 


spective programmer’s attention to this problem. 

Input and output. The program library is especially helpful in 
providing subroutines for putting data into the computer and getting 
results out. The computer’s instructions for transmitting data between 
the storage unit and the input and output devices are complicated and 
usually must be accompanied by quite a bit of internal processing to 
get the data in the proper form. Hence, each computer installation pre- 
pares a set of standard input and output subroutines for use by all pro- 
grammers. Data are put into the computer by means of one of the input 
subroutines, and an output subroutine will be used to get the results out 
of the computer. Because the subroutines are quite flexible, they require 
extensive calling sequences to specify the form and quantity of input or 
output. Learning how to program for input and output becomes a matter 
of learning how to use the subroutines, rather than how to use the input 
and output instructions on the computer. 

As an example, suppose that a list of 40 numbers is to be read into 
the computer from a deck of cards. Each card contains 1 five-digit 
number, in card columns 11 to 15. We would use a card input subroutine; 
the calling sequence would specify the number of cards to be read in, the 
location of the number on the card, and the internal storage location 
that is the destination of the first number in the list. As usual, many 
variations are possible: there might be several numbers on each card; 
the total number of numbers might not be fixed but rather might be 
specified by the first card in the input deck; the numbers might be placed 
in different locations in storage; etc. The details of all these possibilities 
are specific to the particular subroutines and can only be acknowledged 
here. 


Illustrative example: integer print-out subroutine. A distribution of 
word lengths in a passage of text lias been tallied by the computer (see previous 
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illustrative example) and must be printed out. The tallies are stored in a block 
of 21 registers, the first of which has the symbolic address DIST. The register 
DIST contains the number of 0-letter words (presumably 0), the next successive 
register contains the number of 1-letter words, the next register has the number 
of 2-letter words, ..., and the last register the number of words having 20 or 
more letters. 

The subroutine that will be used to print the results is designed only for 
decimal-integer print-out and has the symbolic name IPRINT. It sets up and 
prints one line on a 120-column line printer. Any number of integers may be 
printed on the line. Its symbolic calling sequence is as follows: The subroutine 
is entered by SET A AND JUMP TO IPRINT. The subroutine exits to 1:A 
if it detects an error in any of its inputs (such as a request for printing in more 


Table 4-2. Symbolic program for distribution print-out 


Location | Content of register Comments 


Program 
LOAD B FROM TWENTY | Start at last in 
distribution 
PRLOOP | STORE B IN STUB Set number for start 
SET A AND JUMP TO IPRINT | Enter subroutine 
HALT Error return from 
subroutine 
- STUB First integer is in STUB 
+7 First integer goes in 
column 7 
— DIST:B Second integer is in 
DIST:B 
—25 Second integer goes in 
column 25 
INDEX B BY MONE Subroutine returns here 
JUMP IF B PLUS TO PRLOOP | Go to print next line 
(Rest of program) 


Constants 

TWENTY +20 Preset for loop 

MC INE — 1 Index for loop 

STUB — Storage for stub 

Data 

DIST Block 21 (Assembly program sets 

aside 21 registers) 

Subroutine 


SUBROUTINE IPRINT | (Assembly program 
inserts IPRINT) 
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than 120 columns). The next two registers in the calling sequence specify the 
location in storage and the desired position on the printed line of the first 
integer to be printed. The storage location may be a standard address or an 
address with an index register. In the latter case, the subroutine will calculate 
the effective address. Any index register can be used except A, which is used 
for subroutine linkage. The location on the printed line is given with reference 
to the units digit in the integer. Additional integers to be printed are specified 
by additional pairs of registers in the calling sequence. The end of the calling 
sequence is indicated to the subroutine by making the number in the last 
register negative rather than positive. The subroutine then prints the line it has 
set up and exits to the following register. 

The routine to print out the distribution table is shown in Table 4-2. A sample 
print-out is shown in Table 4-3. A loop is needed because 21 lines are to be 


Table 4-3 Sample output from program in Table 4-2 


Word Number 
length of words 
20 1 
19 0 
18 0 
17 3 
16 0 
15 1 
14 0 
13 0 
12 4 
11 7 
10 6 
9 13 
8 20 
7 31 
6 55 
5 84 
4 122 
3 150 
2 73 
1 30 
0 0 


printed. Two numbers are printed on each line: the stub, showing the number 
of letters; and the main entry, showing the number of words having that many 
letters. The table will start with a word length of 20 or more and go down to a 
word length of 0. Index register B will count the loop and also serve as the source 
of a number for the stub. The subroutine is entered in the specified manner. 
If anything goes wrong, we want to stop immediately; so we put the instruction 
HALT in the register to which the subroutine returns if trouble occurs, ‘I he next 
two registers in the calling sequence indicate that the integer in STUB is to be 
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printed with its rightmost digit in column 7 on the paper. The next two registers 
indicate that the integer in the cell whose effective address is DIST:B is to 
be printed with its rightmost digit in column 25. The 25 is negative, indicating 
that it is the last register in the calling sequence. The subroutine returns to the 
next cell, at which point B is decreased by 1. The print loop is repeated if more 
lines remain to be printed out. 

This simple routine is an abstraction from a real print-out routine for the 
IBM 704/709/7090. The real routine has many more options, the details of 
which require material discussed in Chaps. 6 and 7. The routine can, for example, 
print out the heading shown in Table 4-3; it can also handle numbers that are 
not integers. It does, however, have the main features illustrated here: the calling 
sequence must show the location in storage and the required position on the 
printed line for each item; addresses with index registers may be used; only one 
line is printed; the calling sequence has a variable number of registers. 


Loading. The use of library subroutines explains how data and 
parameters can be read into the computer under the control of the 
program itself. One big question remains: How does the program get 
into the computer in the first place? The answer is complicated, since 
a three-step process is generally required. First, the program is written 
on special coding sheets in symbolic form and is typed onto cards or tape. 
Next, the computer assembles a digital form of the program from the 
cards or tape, with absolute values assigned to all symbolic addresses, 
and with the numerical codes for all computer instructions inserted. 
The output of the assembly program is a deck of cards containing a 
compact digitally coded form of the original program, together with an 
assembly listing. A special library program called a loader is used to 
read the assembled program into the computer when it is to be run. 

Why the extra steps? Why not have the assembly program also load 
the program so that the computer can execute it then and there? The 
answer has to do with efficiency. The assembly process takes much more 
time than loading the assembled program. Presumably the program will 
be run several times, with different batches of data, and so it is more 
efficient to assemble the program only once. Also, assembly sometimes 
uncovers errors in the original program such as undefined symbols or 
omitted addresses, and these errors must be corrected before the program 
can be run. Finally, at large computer installations, it turns out to be 
convenient to assemble many programs at the same time by means of a 
production-line operation. 

Monitors. The continued pressure for efficiency has resulted in a 
further consolidation of effort. In place of a simple loader, many in- 
stallations use an elaborate program called a monitor to load the user’s 
program. The monitor can load and run several programs, one after the 
other, without any intervening manual steps. The assembled programs 
to be run are transeribed off line onto a single magnetic tape. The 





70 Programming 


monitor reads in each program, starts the program, and when the 
program is completed, replaces it with the next program in line. Monitors 
have built-in input and output routines and built-in debugging routines 
that can be used by the programs being monitored. Each computer 
installation generally has its own monitor with its own special features 
that the programmer must learn about when he goes to use the machine. 
Here we merely warn him of the possible existence of a monitor with its 


' own special ground rules. 


4-4 Debugging 


So many details of programming require complete accuracy that errors 
are bound to occur. A new program seldom works properly the first time 
it is run on the computer. Between the initial programming and the final 
production run comes the inevitable task of finding the errors, or “bugs,” 
in the program. 

Most errors occur in jumps and loops. An instruction that was inserted 
after the program was first written may disturb some addresses so that 
the flow of the program is interrupted. An incorrect jump may send the 
computer to get its next instruction from a register containing a constant 
or an irrelevant instruction. A loop may be traversed once too often or 
once too seldom. It is very easy to be off by 1 in the count, because of the 
computer’s unnatural desire to start counting at 0 instead of at 1. The 
conditional jump that tests the end of the loop may be reversed, so that 
the loop is traversed only once, or the condition may be tested wrong, 
so that the computer never gets out of the loop. This last malady, known 
as an e loop, is very common and is easy to detect while the 
ae The rhythmic cycling of the computer’s indicator 
lights shows with embarrassing clarity that the computer is spinning its 
electronic wheels. 

Another likely source of errors is the programmer’s own cleverness. 
Programmers are strongly tempted to make their programs as short as 
possible by taking advantage of some of the computer’s special features 
and by cleverly combining the standard operations. Once in a while such 
tricks are valuable, but generally the anguish they cause in debugging 
far outweights the satisfaction of a clever coup. Computer buffs take 
pride in finding ways to shorten a program, and this activity is useful to 
the novice as an exercise in understanding computer operations, but as a 
practical matter the time saved by eliminating one or two instructions 
from a program is much less than the computer time wasted in debugging 
the more complicated, shorter program. Removing one instruction from 
a program loop that will be executed a million times saves only a few 
seconds on a high-speed computer, whereas one extra debugging run 
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may cost several minutes of computer time, to say nothing of the man- 
hours spent in tracking down the error. 

Standard procedures have been devised to cope with expected errors 
Generally, some test data are used for which the result to be eornputed 
is known in advance. A comparison of the actual and expected outputs 
will then show whether or not the program seems to be working properly 
Also, if the computer stops before finishing the program, or if it is lognine 
and must be stopped manually, a record is made of the entire contents 
of the computer—all storage registers, the accumulator, index registers 
etc. This record, called a dump, is made by a special computer sroenaah 
and can be used in tracking down the source of the trouble. If the com- 
puter has many thousands of registers, only some of which are used by 
the program, a selective dump is made, recording only the registers of 
interest. 

For very elusive errors, special library routines can be used to record 
more extensive information. A record called a trace can be made of the 
contents of all relevant registers on every computer cycle, like those 
shown in Figs. 3-5 and 3-7. Producing such records requires - extrava- 
gant amount of computing time, and so compromises are often made 
1 he debugging program can be set to make a trace only when certain ieey 
instructions are executed. Dynamic evidence of the progress of the 
computation can also be provided by selective dumps made periodically 
during the execution of the program. Other possibilities depend on the 
versatility of the library debugging routine being used. | 

Debugging can be a traumatic experience for the novice. Most of us 
‘ure not prepared to have our own errors thrown up at us so tactlessly 
Moreover, each bug can seem very trivial when it is finally incovened: 
and the how-stupid-of-me feeling can be very humbling. Professional 
programmers are accustomed to their own fallibility and face the task 
of debugging with equanimity. They realize that programming errors are 
nobody’s fault but are the natural consequence of being human. In fact 
many programmers take pleasure in solving the exciting and inbrieate 
puzzles that result from their own mistakes. 

Debugging is a much more difficult problem than programming be- 
cuuse the clues are often quite remote from the actual trouble. When 
confronted with a dump, the programmer must first determine roughly 
What part of the program contains the bug. Then he must mentally trace 
(he computer’s actions in this part of the program, to determine just why 
the computer did what it did. If this can be determined, the bug will be 

obvious. If a bug is found but the programmer still dies: not understand 
the computer’s behavior, the program probably contains other bugs 
When a bug is very elusive, there is a strong temptation to believe that 
the program is correct but the computer erred. It is a mistake to yield 
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to this temptation. Almost always the program is at fault, not the 

er. 
ae can be aided greatly by taking special precautions in 
writing the program, first, to reduce the chance of an error, and, second, 
to facilitate locating potential errors. A few suggestions will be listed 
a opade the program into small, logically distinct segments. Pro- 
gram each segment separately, leaving the results in separate registers 
wherever practical. That is, make each segment as independent as 
arn "ike straightforward procedures; avoid shortcuts. Let the program 
mirror the logical procedures you would naturally use, rather than some 
intricate clever process that might save a little time or space. 

3. Annotate the program carefully. It is surprising how quickly one 
forgets his own train of thought in a computer program. After only a few 
days, a programmer may fail to recognize his own program. A few oe 
ments, punched in the program cards so that they will stay with the 
program, can be extremely helpful in recalling the details ofa caine 

4. Try to anticipate all possible changes that might later be made 
in the program, and include provisions for those modifications, carefully 
annotated, of course. Changing a program that was not meant to be 
changed can result in so many bugs that starting over from the beginning 
is the only cure. Specifically, the number of times each loop is traversed 
should be variable, and a separate storage cell should be set aside for 
each index limit, even if several cells will thereby contain the same 


constant. 


— 4-5 Computer SOP 


At this point, a recapitulation is in order. A step-by-step chronicle of 
the preparations of a program will delineate the standard operating 
procedures (SOP) that are followed in using a computer and will help to 
put the programming task in proper perspective. 

1. Planning. A flow chart is prepared, if the program is compli- 
cated. Sometimes the programmer plans the program without actually 
drawing out a flow chart. 

2. Coding. The programmer writes his program on special coding 
sheets, taking care to follow the rules of the computer and the symbolic 
assembly program. Whenever possib'e, the programmer uses subroutines 
available in the library. In some cases, the programmer writes his own 
subroutines. 

_ 8. Punching. The programmed instructions from the coding sheets 
are typed onto punched cards, punched tape, or magnetic tape by means 
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of a special machine with a typewriterlike keyboard. This is done either 
by the programmer or by a specially trained clerk. A printed list of the 
program is usually prepared from the cards or tape, as a record for the 
programmer. 

4, Assembly. The assembly program is loaded into the computer, 
and the cards (or tape) from step 3 are read into the computer by that 
program. The assembly program produces an assembly listing and a tape 
or a deck of cards that is ready to be read into the computer. If any 
errors are found, a list of these errors is printed; the programmer must 
correct his original program and reassemble it. After assembly the 
programmer has two versions of his program—the symbolic version 
produced at step 3 and the assembled version produced at step 4. He 
keeps the former, in case changes must be made, but the latter will be 
read into the computer. 

5. Data preparation. Perhaps concurrently with steps 1 to 4, the 
data and/or parameters that are to be read in by the program must be 
transferred to cards or tape, following the procedure in step 3. In 
addition, it is wise to prepare some specially contrived data and/or 
parameters, for which the correct computer output is known in advance. 
These will be used to test, or check out, the program. 

6. Check-out. The assembled program and the test data are 
brought to the computer. A special utility program, either a loader or a 
monitor, is put into the computer; this program automatically loads the 
user’s assembled program and automatically starts it at the place 
indicated by the programmer. The program begins; during its operation, 
the program causes the computer to read the test data. Hopefully, the 
computer produces the desired output and stops at the expected place. 
Usually something will go wrong, and one of the debugging procedures 
must be used. 

7. Production. When the program appears to be working correctly 

perhaps after several tries—the procedures of step 6 are followed, 
with the real data being used, and the long-awaited results are pro- 
duced. 

8. Write-up. Concurrently with the above steps, the conscientious 
programmer prepares a write-up that describes how to use his program, 
so that at a later date he or someone else will have no trouble in using 
the program again. The write-up supplements the comments in the 
symbolic program cards, but it should be complete and detailed. All 
conventions and special tricks used in the program should be stated, 
und rules for the input and output should be very explicit. Even though 
the programmer may have no intention of filing his program in the 
library of programs at the computer installation, there is always a chance 
that the program will be used again. 
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Problems 


1. Prepare a symbolic version of each of the problems in Chap. 3. 

2. Obtain the specifications for an input and an output subroutine. Prepare 
calling sequences for these routines to perform the input and output of any 
program in this book. 

3. If you intend to use a particular computer and/or a particular computation 
center, find out the coding rules for the conversion or assembly programs 
in use. Obtain coding sheets. Find out what library subroutines and routines 
are available. Write a small program including input and output, and carry 
through all the steps in the computer SOP. Before running the program 
on the computer, estimate how much computer time will be required. After- 
ward, check your estimate. 
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Programming languages 


\ \ / rrv1NG digital-computer programs in the language of machine 
instructions can be very tedious. Despite the convenience of the 
symbolic assembly program and the library of subroutines, programming 
remains a chore. The programmer must pay close attention to a myriad 
of details and is forced to think about his problem in the small units that 
single computer instructions represent. To write a large and complex 
program in such small units is a monstrous task. Consequently, special 
programming languages have been developed so that operations re- 
quiring large amounts of computation can be expressed in a small 
number of words or symbols. With the programming language, complex 
computer programs can be expressed in units that are convenient for the 
programmer, and not in the units imposed by the machine design. 

The development of programming languages has been accompanied 
by a growth in the capacity of computing machines. This is not a 
historical accident—each has stimulated the other. The special languages 
have facilitated the writing of programs that taxed the capacities of 
existing machines, and the potentialities of increased machine capacity 
have best been realized by complex programs expressed in the more 
powerful languages. Faster, larger machines go hand in hand with more 
powerful programming languages. 

The first languages to be developed were in the field of mathematics. 
l'OrTRAN (an acronym for FORmula TRANslator), Ir, and Mars- 
MATIC all permitted complex mathematical expressions to be stated 
naturally, in some form of algebraic notation (see Backus, 1959; Hopper, 
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1959; Taylor, 1961). In addition, these languages provided versatile 
control statements to facilitate the expression of iterative processes and 
other computational-routines. Each language requires a computer pro- 
gram called a compiler>to translate the notations permitted by the 
language into machine instructions for the computer. Today, every 
large computer has an algebraic compiler for translating mathematical 
expressions into machine code. Using such a compiler, many scientists 
who cannot take the time to learn the details of machine programming 
are able to put their problems on a computer. 

The profusion of algebraic compilers has led to an effort at standard- 
ization. An international committee of computer experts met and 
produced the specifications for an all-inclusive algorithmic compiler 
called Atcou (for ALGOrithmic Language). Unfortunately, the com- 
mittee’s efforts to achieve generality and precision resulted in a very 
complex language designed for the professional programmer rather than 
the sometime computer user (see Naur et al., 1960, and Bottenbruch, 
1962). ALGoL appears to be popular in Europe but has not yet been 
widely used in this country. 

Recently, programming languages have been developed for a wide 
variety of problems. Conon (COmmon Business Oriented Language) is 
designed to handle business problems of inventory and bookkeeping 
which require the computer to organize, maintain, and sort large files of 
information (Sammet, 1961). Military command and control systems 
which involve large amounts of both computation and filing require 
compilers combining the features of ALcou and Cosox; Cup (Isbitz, 
1959) is one of several such compilers. In the field of linguistic processing, 
which includes mechanical language translation, Comir has been de- 
signed to allow the words in the text to be the units that the program 
processes. ApT (Automatically Programmed Tool; see Ross, 1959) was 
designed to facilitate the automatic control of machine tools. The com- 
puter simulation of intelligent_behavior is conveniently done in a list- 
processi nguage such as ‘TpL-v ormation Processing Language 
Five) Tas as Processor)-or{FiLpxi (Fortran List Processing Lan- 
guage), \whi¢h organize memory in special associative structures. List 
processing languages are discussed in the second part of this chapter. 

The fact that programming languages are problem-oriented rather 
than machine-oriented has three important advantages in addition to 
programming convenience. First, persons who have problems but do 
not have a detailed knowledge of the computer can use the computer to 
solve their problems. Programming languages are much easier to learn 
and understand than are the details of computer operation. Second, there 
is an immense psychological advantage in being able to express programs 
in the units in which a person naturally thinks about his problem. The 
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programs are more easily comprehended and remembered when ex- 
pressed in appropriate units. Third, programs written in a programming 
language are not tied to a specific computer but can be executed on any 
computer for which a translator exists. Programs written in ForTRAN, 
for example, can be run on most of IBM’s large computers as well as on 
many other computers. A simplified form, called FortTransit, is avail- 
able for the IBM 650. 

For every programming language used on a particular computer there 
must be a special computer program, written in the language of the 
computer, that translates the statements of the programming language 
into machine instruction. The translation can be done in two funda- 
mentally different ways. Compilers translate each statement into a 
sequence of machine instructions, in symbolic form, according to a 
standard algorithm. Thus the expression (C = A + B) might be trans- 
lated into the three instructions LOAD FROM A, ADD CONTENT 
OF B, STORE IN C. Then a symbolic assembly program transforms 
this sequence into absolute machine language, punching out a digitally 
coded deck or tape. To run the program, the deck or tape is fed into the 
computer by a loader. Thus, when the compiled program is run, the 
computer contains only the translated version of t rogram, in the 
form of a sequence of machine instructions. (Imterpreters,\on the other 
hand, code each statement in the programming Ianguage in a form con- 
venient for computer storage, so that when the program is to be run, a 
coded, but untranslated, version of the program is stored in the com- 
puter. The interpreter decodes and translates each statement when it is 
to be executed. The translation and execution phases are completely 
interwoven. 

Since an interpreter must translate each statement every time it is 
executed, statements that are parts of loops are translated very often. 
Consequently, interpreted programs are executed much more slowly 
than compiled programs. On the other hand, if a program is to be 
changed fairly frequently, then omitting the intermediate translation 
step of a compiler is often a convenience. Also there may be a storage 
advantage, since every statement in the programming language results 
in several machine-language instructions. A program in the original 
language has many fewer symbols than its machine-language translation. 
Of course the interpreter must also be stored, but it can happen that the 
interpreter and the program together take less storage than the corre- 
sponding compiled program. Despite these conveniences, computer ex- 
perts usually have a strong preference for compilers because of the much 
wreater efficiency in running the completed, debugged program. 

Debugging a compiled program presents special problems. A compiler 
is a one-way communication link—it cannot translate machine language 
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back into programming language. The programmer may communicate 
with the computer in the programming language, but the computer 
cannot use the same language in communicating with the programmer. 
The standard debugging procedures, such as dumping and tracing, pro- 
duce records in machine language and are of relatively little value to a 
programmer who knows only the programming language that he is using. 
Consequently the programmer must take special pains to include de- 
bugging provisions within his program. That is, he must arrange to print 
out volumes of intermediate results for checking purposes, so that he can 
determine just where his program goes wrong. This is easy enough to do 
but requires foresight as well as experience in the kind of errors to 
anticipate. 

Some interpreters are no better than compilers in communicating 
with the programmer. Others have extensive provisions for communi- 
cating in the programming language. Interpreters have access to the 
coded program, in the programming language, which is an immense ad- 
vantage in providing two-way communication. Tracing, for example, 
can be done in the programming language; the results of each interpreted 
instruction can be printed. 

Of course, the best debugging technique is to avoid errors in writing 
the program. The suggestions given in Chap. 4 for machine-language 
programs also apply to programs written in a programming language. 
Divide the program into easily managed, independent parts; use pro- 
cedures that seem straightforward to you, rather than clever tricks; 
annotate carefully; anticipate changes. 

Using a programming language requires careful attention to details. 
Unlike natural languages, programming languages allow no ambiguities 
or imprecise statements. Every symbol has an exact, unambiguous mean- 
ing to the compiler. The compiler treats each statement as a string of 
symbols and interprets this string according to fixed rules. One too few 
parentheses in an expression, or one too many, will cause the statement 
to be rejected. A forgotten or misplaced comma will often cause trouble. 
Nevertheless, programming languages are an immense improvement 
over machine language. A programmer who follows the rules carefully 
can write very complicated programs with relative ease. 


5-1 Algebraic Compilers 


Almost every computer has associated with it a compiler for translating 
statements written in some form of algebraic notation. These program- 
ming languages are all very similar to one another; a programmer who 
knows one can very easily learn to use any of the others, In this chapter, 
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specific examples will be drawn from the most widely used algebraic 
compiler, ForTRAN.! 

Arithmetic statements. The basic function of algebraic compilers 
is to translate a mathematical computation stated in a very natural way, 
such as 


RESULT = X + Y—-Z 


into a sequence of machine-language instructions similar to that of 
Table 3-1. The program statement is very similar to an algebraic 
equation; the main difference is in the meaning of the equals sign. In 
algebra, = states a fact, namely, that two expressions are equivalent. 
In a programming language, = indicates a process, namely, ‘Replace 


Ors value of the quantity on the left by the value of the expression on 


the right.”’ The two meanings are not always equivalent. In indexing, for 
example, we shall sometimes need to increase a quantity by 1. The 
following statement, while not an algebraic equation, is legitimate for 
the compiler: 


X=X+1 


‘The meaning of this statement is ‘“The new value of X is the old value 
of X plus 1.” 
For more complicated mathematical formulas, a parenthetical no- 


tation is used to denote the order of calculation. For example, the 
formula 


—a-b 
cd 


is written in ForTRAN notation as 
R = ((S/F) — (A*B))/(C#D) 


where multiplication is indicated by asterisks and division by slashes. 
‘The computer, like the high-school student, will perform the calculation 
from the innermost parentheses to the outermost parentheses. Thus, 
S/I and A*B will be calculated first. Then their difference will be ob- 
(uined as the numerator. The denominator is calculated next, and finally 


''The discussion of Fortran is based on Reference Manual, 709 FORTRAN Auto- 
matic Coding System for the IBM 709 Data Processing System, Copyright 1959 by 
International Business Machines Corporation, which is used with permission. The 
present exposition is the author’s own account of Forrran, and all responsibility 
for errors rests with him, A complete introduction to Fortran can be found in 
MeCracken (1961), 
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the numerator is divided by the denominator to obtain R. In Fortran, 


‘\/ynd in most other compilers, multiplication and division take precedence 
over addition and subtraction, so that the above formula may also be 


written as 
R = (S/F — AxB)/(C*D) 


Within the parentheses in the numerator, the multiplication and division 
will precede the subtraction in the compiled program. 
The compiler contains a number of built-in subroutines for calculating 
simple trigonometric functions, square roots, logarithms, ete. In ad- 
dition, the programmer can provide special subroutines of his own to 
compute functions that he intends to use frequently. These functions can 
then be inserted as parts of arithmetic expressions. For example, the 
formula for the standard deviation can be written algebraically as 


where a is the mean and s is the sum of squares of a set of f numbers. 
The Fortran expression for this formula would be 


SIGMA = SQRTF (S/F — A*#2) 


In this expression the double asterisk indicates an exponent, SQRTF 
indicates the subroutine for the square-root function, and SIGMA is the 
name by which the programmer chooses to symbolize the standard 
deviation. The compiled program will first calculate the value of the 
expression within the parentheses and then will use the indicated sub- 
routine to find the square root of the value of the expression. 
Fortran recognizes two different kinds of numbers. Floating-point 
numbers are used in most calculations, while integers are used for index- 
ing operations, (A floating-point number is a special digital representation 
that allows thé-Computer_to keep track of the decimal point auto- 
matically in all calculations.)Some digits in the number are used to en- 
code the value of the number, and other digits encode the location of the 
decimal point. In decimal notation, the floating-point representation of 
a number is a fraction between o and 1 and a scale factor. Thus, 250 is 
0.25 X 10%, 0.025 is 0.25 X 107, —2.5 is —0.25 X 10. The computer 
stores both the fraction and the exponent. There is an equivalent no- 
tation for binary numbers; the details of both notations are discussed in 
Chap. 6. At present it is sufficient to note that the use of floating-point 
numbers relieves the programmer of essentially all concern for the size of 
the numbers being manipulated or for the decimal point. However, the 
compiler must also provide for calculations with integers. The calcu- 
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lation of numbers that are to index a computation loop, for example, 
must be done with integers. For example, if some process must be done 
for half of n numbers, the integer n (which we assume to be even) must 
be divided by the integer 2 to obtain the upper limit for the index in the 
processing loop. In general, numbers destined for index registers are 
treated as integers. 

With two number systems available in Fortran, some means must be 
provided for keeping them straight. Rules are needed for named quanti- 
ties, such as RESULT, and for specified numbers, such as 75. Lhe rule 
for specified numbers is that a floating-point number must fave a 
decimal point and an integer must not)JThus, 75. is treated as a floating- 
point number, and 75 is treated as an integer. A named quantity that is 
to be treated as an integer must have a name starting with one of the 
letters I, J, K, L, M, and N, for example, NCASE, IMAX, I, ete. A 
name starting with any other letter, for example, X, SIGMA, etc., is 
treated as a floating-point number. ForTRAN does not allow both integers 
und floating-point quantities in the same expression. All quantities must 
be of the same type. On the other hand, quantities on different sides of 
{he = sign in an algebraic statement may have different types. In this 
tase the compiler performs the necessary transformation after calcu- 
lating the expression. Thus, the statement 


—_— 


xX = NMAX —N 


means “calculate the difference of the integers NMAX and N, convert 
the resulting integer to a floating-point number, and store it as the value 
of X,” 

Control statements. A ForTrRAN program is a sequence of state- 
ments that are evaluated one after the other in sequential order unless 
altered by a control statement. Statements may be given arbitrary 
statement numbers to identify them for the purpose of controlling the 
flow of the program. The simplest control statement is an unconditional 
branch, GO TO N, where N is the number of some statement. A branch 
can also“*be made conditional on the numerical value of some quantity. 
l'ORTRAN provides a three-way branch that depends on whether the 
value of an expression is less than zero, equal to zero, or greater than zero. 
‘lhe conditional branch statement is the word IF followed by an expression 
in parentheses, followed by three statement numbers, i.e., IF(S) Nu, 
Ny, Ng. If the expression § is less than zero, the program goes to state- 
mont Nj; if the expression equals zero, the program goes to statement 
Ny; and if the expression is greater than zero, the program goes to 
alatement Ns. Thus, in the statement 


IF (X-A) 3, 7, 22 
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the program goes to 3 if X is less than A, to 7 if X is equal to A, and to 
22 if X is greater than A. 

Consider the example from Sec. 3-2. Suppose that we wish to calculate 
X + Y — Z but to limit the result to no more than 75. The program in 
Table 3-4 becomes the Fortran language program in Table 5-1. If the 


Table 5-1 Example of Fortran program with a conditional statement} 


RESULT = X+ Y—Z 
IF (RESULT — 75.) 1, 1, 2 
2 RESULT = 75. 
1 [Next statement in program] 


} Equivalent to Table 3-4. 


calculated value of RESULT is less than 75, the expression (RESULT 
— 75.) will be less than zero and the program will go to statement 1. 
The program will also go to statement | if RESULT is equal to 75. On the 
other hand, if RESULT is more than 75, then the quantity (RESULT 
— 75.) will be positive and the program will go to statement 2, after 
which it will continue with the next statement, which happens to be 
labeled 1, in the normal flow of the program. Note that the statement 
numbers are merely labels and need not be assigned in numerical order. 

Subscripts. In algebra, subscripts are used to denote particular 
numbers from a set. Thus we might use x to designate a set of 40 numbers 
and 2; to designate the ith number in the set. Consequently algebraic 
compilers must permit the use of subscripts. ForTRAN uses parentheses 
to denote subscripts: x; is written X(I). Note that this does not mean 
X times I, which would be written X+I or X*(I). Up to three subscripts 
are permitted, that is, X(I), Y(I, J), Z(, J, K), but not W(I, J, K, L). 

To indicate the summation of 40 numbers in algebraic notation, we 
might write 

40 
s= y Li 
i=l 


which is read “s is equal to the sum, from 7 = 1 toz = 40, of x;.”’ In 
Fortran, the arithmetic statement 





Cs0M = SUM + x 
™ 


must be performed successively for values of I from 1 through 40, after 
first setting the value of SUM. to 0, One possible program to do this is 
shown in Table 5-2. First the value of SUM is set to 0, and the index I 
is set to 1. The first number is added to SUM, and one is added to I, 
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Table 5-2 Fortran program to sum 40 numbers 


SUM = 0. 
I=1 

5 SUM = SUM + X(1) 
I=I+4+1 


IF (I — 40) 5, 5, 6 
6 [Next statement in program] 


Then, if I is less than or equal to 40, the loop repeats, but if Lis more than 
40, the loop is finished and the program continues. 

DO statements. The indexing loop of Table 5-2 is so typical of 
numerical calculations that algebraic compilers generally provide a 
special indexing statement to simplify the programs. In Fortran the 
DO statement serves this function. This statement, which precedes the 
loop, specifies the extent of the loop, the index being changed, the limits 
of this index, and the index increment for successive traversals of the 
loop. Thus the statement 


— 


— ~ DO5I=1,40,1 


means “Do the following statements down through statement 5, for 
successive values of I from 1 through 40, incrementing I by 1.”” When 
the increment is 1, it need not be written, since the compiler assumes an 
increment of 1 unless otherwise specified. The revised addition program 
is shown in Table 5-3. The DO statement causes the compiler to put some 


oa — _ a oe 


‘Table 5-3 Fortran program with DO loop 


SUM = 0. 
DO 51 = 1, 40 
5 SUM = SUM + X(I) 
[Next statement in program] 


index-setting instructions in the machine-language program at the start 
of the loop and some indexing and testing instructions at the end of the 
loop. 

An important feature of the DO statement is that the limits do not 
lhuve to be fixed when the program is written. The limits may be vari- 
ables whose values may either be read into the computer at the start of 
the program or be calculated by the program. Thus, in Table 5-3, 40 
could be replaced by N, where N is some parameter that is read in with 
(he data or is some value calculated by preceding program statements. 
‘Thus, very general DO loops can be written easily. 

Specification statements, When a quantity has a subscript, the 
compiler must have some way of knowing the maximum value of that 
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subscript so that a block of registers may be set aside for the set of 
quantities. Thus, if X(I) may refer to any one of 40 numbers, a block 
of 40 registers must be set aside for X. In an intercorrelation problem 
involving 30 variables, R(I, J) may refer to any of the 900 intercorre- 
lations. Every ForTRAN program that uses subscripts must start with a 
DIMENSION statement. This statement is not a part of the operating 
program, but it specifies the storage requirements by indicating to the 
compiler the maximum size of the subscripts for all variables having 
subscripts. Thus, 


o 


DIMENSION X(40), R(30, 30) 


means “Set aside a block of 40 registers for X; set aside a block of 900 
registers for R, to be treated as a 30 X 30 matrix.” 

Input-output statements. Input and output are accomplished by 
READ, WRITE, and PRINT statements. Each statement specifies the 
input or output unit involved, a list of quantities to be processed, and 
the number of an associated format statement, which in turn specifies 
the precise manner in which the quantities appear in the input or output 
medium. For example, the following statement will read in a list of 40 
numbers from cards: 


READ 100, (X(1), I = 1, 40) 
100 FORMAT (F6.2) 


{ This is interpreted as ‘‘Read from cards, according to the format speci- 
fication numbered 100, the list of quantities x;, for i from 1 through 40.” 
The associated format specification indicates the exact columns on the 
card where the number appears. In this case there is one decimal number 
per card in the first six columns, with two places to the right of the 
decimal point, the decimal point thus being between columns 4 and 5. 
The details of format specification are both special and complicated and 
may be found in programming manuals. 

Diagnostics. The rules for writing ForTRAN programs are specific. 
In particular, commas and parentheses must be used only in certain 
ways, and the format of each statement is fixed. The ForTRAN compiler 
checks cach statement and prints a diagnostic statement if the rules 
have not been followed, indicating as best it can what is wrong. The 
compiler also examines the program for more complicated errors. For 
example, the flow of the program is examined; if there is a statement that 
the program can never get to, under any conditions, this fact is noted by 
a diagnostic statement, The sophistication of the diagnostic routines is a 
creat aid in debugging and helps to avoid having to debug the compiled 
program in machine language, ° 
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Other features. Some features of Fortran have not been discussed 
here for lack of space. Fortran includes facilities for compiling sub- 
routines separately and for incorporating them into other programs. 
Additional control statements are available for n-way jumps. A number 
of built-in functions are available, such as choosing the largest value from 
a designated set or determining the absolute value of a quantity. Many 
other details have been omitted in the interests of clarity. But enough 
has been presented so that some powerful programs can be written. 


Illustrative example: standard scores. A program is required to calculate 
the standard scores on any test. The inputs are to be the raw scores, together 
with a parameter N, specifying the number of raw scores. The program is to 
be able to handle any number of scores up to 1,000. The output is to include 
the input parameter N, the mean and standard deviation of the raw scores, 
and a list showing each raw score with its associated standard score. A complete 
FortRAN program is shown in Table 5-4. 


Table 5-4 Fortran program for standard scores 


. 





The DIMENSION statement tells the compiler to set aside two blocks of 
1,000 registers, one for the raw scores, labeled X, and one for the standard 
xcores, labeled 'T. The next statement starts the program by reading in the raw 
scores from an input tape. (Initially the scores must be punched on cards and 
then put onto tape by a card-to-tape machine.) According to the statement, 
the input tape is to be on unit 2, the data format is specified by statement 100, 
and the input is to consist of the parameter N, followed by the raw scores X.. 
According to the format statement, the details of which we shall omit here, N is 
road in as an integer, but the X; are translated to floating-point numbers in the 
computer, Since we shall need N in our floating-point calculations, we define 
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a new quantity, FN, which is the floating-point equivalent of the integer N, 
recalling that, in ForTRAN, quantities labeled with a name having I, J, K, L, M, 
or N as the first letter are integers and all others are floating-point. The state- 
ment FN = N means “Set the floating-point variable FN equal to the floating- 
point equivalent of the integer N.” 

Following this conversion, two presets are made: S and SSQ are set equal 
to zero. Then the sum and surr of squares of the scores are calculated by a DO 
loop. The DO statement means ‘Do the following statements down through 
statement 10, starting with I = 1, and ending with I = N, incrementing I by 
1 on each cycle” (the incremert is 1 if not otherwise specified). S accumulates 
the sum and SSQ the sum of squares. After the first traversal of the loop, 
S = X, SSQ = X.2. After the second traversal, S = X;-+ Xs, and SSQ = 
X,? + X.”, etc. Note that the number of times the loop is traversed depends 
on the input parameter N. 

After the DO loop has been completed, i.e., when it has been traversed with 

=N, the program is ready to calculate the mean and standard deviation. 
We use the symbol FMEAN sather than MEAN because we want the mean 
to be stored as a floating-point quantity. The mean is calculated according to 
the formula 

n 


The formula for the standard deviation is 


1 n 
C= ( } #)—m 
n 
t=1 


The standard scores are calculated by the next DO loop, according to the 
formula 


cae 





This loop contains only statement 20, which is repeated iteratively until I = N. 

The output is written on the tape unit No. 3, according to format 101. The 
quantities to be the output are, in order, the number of cases, the mean, the 
standard deviation, and the list of pairs, X; and T;, for i from 1 to N. (Both the 
WRITE and the associated FORMAT statements are so long that they occupy 
more than one line or card; the rumerals at the left of each overflow card indicate 
that the card is a continuation of the preceding statement and not a new state- 
ment.) 

The compiled program (for tte IBM 7090 computer) occupies 2,147 registers, 
2,000 of which are for storing the scores; 52 are for other storage, and the rest 
is for the main program. Additional.space is needed for the SQRT subroutine 
and the various input-output sabroutines implied by the READ and WRITE 
statements. These subroutines are supplied automatically by the compiler, 
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In a trial run, the ForTRAN program was read in, compiled, and executed with 
a test set of 10 scores, all in less than 1 minute. 


Illustrative example: intercorrelations. Table 5-5 presents a complete 
Forrran program for calculating the means, standard deviations, and inter- 
correlations of the scores on N individuals on each of K variables. The program’s 
inputs and outputs are on tape. A sample output is shown in Table 5-6. 

The plan of the program is to accumulate the sums, sums of squares, and sums 
of cross products as the data are read in, so that the computer needs to store 
only the scores for one individual at a time, not the entire set of N-K scores. 
The accumulating sums will be stored in the registers that later will hold the 
means, sigmas, and intercorrelations. Consequently, storage is required only 
for the K scores of one individual, labeled X; the K means, FMEAN; the K 


Table 5-5 FortTRAN program for intercorrelations 
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Table 5-6 Sample output of intercorrelation program 





i. 


Bes 
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standard deviations, SIGMA; and the K? intercorrelations, R. The program 
in Table 5-5 is set up for a maximum K of 100; consequently a total of 10,300 
storage registers must be set aside for the subscripted variables. 

The program first reads in the parameter K. It tests K for size; if K is more 
than 100, an error message is printed out and the machine stops. The error 
message reminds the user that there is a limit on K and that he will have to 
alter the program to accommodate this problem. If K is acceptable, the computer 
presets the cell that will be used to count the number of cases, as well as all 
the other cells that will accumulate results. 

The program next enters the input loop. On each pass through the loop the 
data for one individual are entered, and their contributions to the various sums 
are processed. The input tape is assumed to have been prepared by an off-line 
card-to-tape machine from a deck of data cards. For each individual there is a 
set of one or more cards containing his data. Each card contains 10 numbers, 
except the last card, which may contain fewer than 10 numbers. The first number 
is an arbitrary case number; then the scores on each of the K variables follow 
in sequence. The case number must be positive and must be greater than zero, 
since the program senses the end of the input deck by the occurrence of a set 
of blank cards for an imaginary individual with a blank case number. The input 
routine converts blank fields to zeros; so a blank case number is a signal for 
the reading to cease. Because of this special way of sensing the end of the loop, 
a DO statement is not used; the sequence is controlled by a conditional IF and 
an unconditional GO TO, 

Note that, since RCI, J) © RG, 1), the sums of cross products are accumulated 
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only for half the correlation matrix. This is accomplished by specifying that the 
limits of J are from I to K, rather than from 1 to K. The same indexing trick 
is used in calculating and in printing out the correlations. Additional time and 
space could be saved by not calculating R(I, I), since by definition R(I, I) = 1. 
However, in debugging it will be helpful to know whether or not the computer 
calculates R(I, I) correctly. If R(I, I) should turn out to be some number other 
than 1, trouble is indicated. 

When all the cases have been processed, the means, sigmas, and intercorrela- 
tions are calculated. Note that the intercorrelations cannot be calculated until 
all the means and sigmas have been calculated. Then the results are written out, 
and the program stops. On the IBM 7090 computer, compiling the program 
takes only a minute or two. The compiled program can calculate the results 
in Table 5-6 in less than 2 seconds, and most of that time is devoted to input 
and output. 


5-2 List processing languages’ 


The use of computers in simulating intelligent behavior and in making 
automata has created the need for a special class of information process- 
ing languages. A typical program for a problem in “artificial intelligence” 
has the following special characteristics: 


1. The program involves manipulating symbols that have nonnumerical 
meaning. 

2. The storage requirements of the program cannot be specified in ad- 
vance; complex data structures are developed as the program 
proceeds. 

3. The relations among the elements of the data are restructured while 
the program is operating. 

4, The problem can be described at several levels of detail and the 
program is stated naturally in hierarchical fashion. 

5. The program will be modified often. 


‘There are at least three information processing languages: IpL, the 
language designed by Newell, Shaw, and Simon (Newell et al., 1961); 
I'LpL, the ForTrRAN list processing language designed by Gelernter and 
his associates (Gelernter et al., 1960); and Lisp, the list processor de- 
signed by McCarthy and his coworkers (McCarthy, 1960). This section 
will discuss the common properties of these and similar languages. 

A list processing language is a way of specifying certain manipulations 
or arbitrary symbols, lists of symbols, and hierarchies of lists called last 
structures. A list processing program to eliminate all occurrences of a 
certain symbol from a given list can be written about as easily as the 
evaluation of an algebraic formula can be written in an algebraic lan- 


‘The material in this section is taken substantially from Green (1961), 
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guage. Before discussing the processes it will be necessary to describe the 
units with which the language deals. 

Symbols. The basic entities of information processing languages 
are not numbers or quantities but arbitrary symbols. The symbols may 
signify anything the programmer wishes—the pieces and the squares on 
a chessboard, the cards in a deck of playing cards, the notes in a musical 
scale, the baseball teams in the American League, etc. All the infor- 
mation that is to be processed must be represented by symbols. The 
programmer specifies the relationships among his symbols by the way 
he puts them together on lists and by the way his program processes the 
lists. 

A list of symbols may also be an entity and may be given a symbol 
as its name. In a program to bid and play the card game of bridge, for 
example, the four hands can be symbolized by N, E, 8, and W. Each 
hand would be a list of the cards in the hand, and so each of the symbols 
N, E, 8, W is the name of a list. In general, any symbo] may be the 
name of a list as well as being a symbol on other lists. Thus, in the 

bridge program there would probably be 

pein Sissad a list of the hands, containing the four 
register symbol Link symbols N, E, 8, and W. 

Lists. The major innovation of list 


473 

485 processing languages is the organization 
of computer memory. Normally, when a 

ie set of symbols is to be stored in a com- 

582 puter, a block of consecutive storage 





registers is set aside and the symbols 
are stored in successive registers. If the 
exact number of symbols to be stored is 
not known at the time the computer program is written, then enough 
storage must be set aside to accommodate the largest possible set of sym- 
bols that is anticipated. Additional blocks must be set aside for sets of 
numbers generated by the program. In complex programs, storage allo- 
cation can be a major difficulty. The concept of list memories was in- 
tended to overcome such difficulties. In a list, each symbol is stored in a 
separate computer storage register, but the registers need not be consec- 
utive. Each register contains two items of information; first, the symbol 
being stored; second, the location of the storage register containing the 
next symbol on the list. This second item of information, called the link, 
serves to keep the list of symbols intact, no matter what particular 
storage registers are used for storing the symbols. To gain access to 
the symbols on a list, the programmer needs to know only the address of 
the beginning, or head, of the list. Each symbol on the list is then located 
by means of the link of its predecessor, The link of the last register on 
the list contains a special termination symbol, say, 0, to signal that 


Fig. 5-1. List of four symbols 
starting in 473. 
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there are no more symbols on the list. The arrangement of a list memory 
is depicted in Fig. 5-1. 

Lists have several valuable properties that stem from using links 
rather than sequential storage registers. For example, a symbol may be 
inserted at any place on the list by merely altering two links. This 
manipulation does not disturb the other symbols. Similarly, symbols can 
easily be deleted from lists or moved from one list to another merely by 
manipulating the linkages. In order to add symbols to lists and to form 
new lists, the list processor will need a group of available storage cells. 
These cells are most conveniently organized as a special available-space 
list. Registers are taken from this 
list as they are needed for storing Data list 
svmbols; other registers are put back 473 
on the list when they are no longer 485 
required for particular storage needs 631 
in the program. Figure 5-2 shows 

sez [| aa | 
how a new symbol is added to a 
data list, by using a cell from the 
available-space list. 

A list of symbols may represent a Available space list 
more complex arrangement of data, 371 
because each symbol on the list may 367 
name a further list. (Each symbol 574 
generally has some sort of prefix to 618 
indicate whether or not it is to be 
treated as naming a further list. The Hig: $2, - Bidabod AB Ww dnperted 3a 
name of a list is the address of its the data list between symbols A2 and 
head.) A bridge deal, for example, 3, by using an available cell. The old 
might be represented as a main list links have been crossed out, and the 
with four sublists. Each sublist would re nie +e stn cpt rae ores 

, ; : ; e change, the cells on the data lis 
Ai Wich 
; ’ 582. The cells on the available space 
sist of the names of the four sublists. list are, in order, 371, 574, 618, etc. 
‘lhe order of the four symbols on the 
main list could indicate the order of bidding and play, for example, N, 
I}, S, W. Interchanging E and W on the list would then correspond 
with Hast and West exchanging hands. 

‘The complexity can be extended indefinitely, because each sublist can 
include symbols referring to still further sublists. The cards in a bridge 
hand, for example, are naturally sorted into suits; the corresponding 
computer representation would have a sublist for each suit. The bridge 
deal would then consist of a main list containing the names of the four 
hands, while each hand would be a list containing the names of up to 
four sublists, one for each suit, In a hierarchy of this sort, the term Level 
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is a natural way to distinguish among the various sublists. The bridge 
deal is at the highest level, the hand at the next level, and the suit 
holdings at the third level. A main list, together with its sublists, their 
sublists, and so on, is called a lest structure. The unit of information in a 
list processing language may be a symbol, a list, or an entire list struc- 
ture. 

One data structure, the description list, is frequently used to charac- 
terize a particular list of symbols. In the bridge example, each hand 
has such attributes as the Goren point count of the hand, the number of 
quick tricks, the biddable suits, and perhaps others. Since the bridge- 
playing program will refer to these attributes often, the attributes of 
each hand are evaluated once for all and each attribute is placed with 
its value on a description list associated with that hand. The description 
list becomes part of the total list structure, so that a bridge deal now 
consists of a main list containing the four symbols for the hands, a list 
for each hand, the description list associated with each hand, and the 
sublists containing the various suit holdings for each hand. In more 
complex programs both the sublists and the description lists can include 
symbols that name further lists, and any list may have an associated 
description list. 

The computer will sometimes be required to build up list structures 
for itself. In a bridge-playing program, for example, the computer may 
have to deal the cards, building up the list structures for a bridge deal 
from an initial list of the 52 cards in the deck. Also, if the computer is to 
learn bridge, it will have to keep records of the rules that it uses in 
various situations, together with the success or failure following the 
application of the rules. For example, the program might initially have 
a large number of possible rules to use in selecting an opening lead. As 
the program played games, its experience might be represented by a list 
of the good rules for finding opening leads. The list might indicate 
priority, rules being moved up on the list when they are successful and 
being demoted when they lead to disaster. Of course, a very sophisticated 
machine would keep several priority lists of opening leads, with de- 
scriptions of the situations in which to use each. 

Processes. In conventional programming, the basic element of in- 
formation is a digital number stored in a register. The available oper- 
ations include ADD, SUBTRACT, STORE, and similar numerical 
manipulations. Machine-language programs combine these operations 
in sequences. In algebraic compilers such as Fortran, the elements of 
information are constants and subscripted variables each representing 
a number or a block of numbers; processes are available for algebraically 
combining these quantities in standard ways, Similarly, list processing 
languages include some basic processes for manipulating symbols, lists, 
and list structures, Processes are provided for comparing symbols, 
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locating a symbol on a list, deleting a particular symbol from a list, 
combining two lists into one, adding a symbol at the end of a list, etc. 
A list processing program is a sequence of these processes. The various 
list processing languages differ in the basic processes that are provided 
for the programmer to use and in the way in which the programmer 
must express his programs for the processor. 

Subroutines are very helpful in conventional programming, and they 
are indispensable in list processing. A list program consists of instructions 
for obtaining the inputs to subroutines, for executing the subroutines, 
and for storing the results. In a complex program the subroutines are 
complex, but the main program is relatively simple. Each subroutine 
represents a unit of processing, i.e., a block in the flow diagram, while 
the program represents only the lines of flow. But each block is itself a 
set of smaller blocks strung together by connections. The program can 
be subdivided into as many stages, or levels, as is convenient. Generally, 
the more levels the better, so that each level can be simple. A list 
program, then, is a hierarchy of subroutines, successive levels becoming 
more and more detailed. In the bridge example, the highest level would 
include routines such as ‘‘Deal the cards,” ‘Conduct the bidding,” 
“Play the deal,’ and “Score the deal.’”’ The block ‘‘Deal the cards” 
would contain subblocks such as ‘Select a card at random” and ‘‘Place 
a card on a hand.” The subblocks in ‘‘Place a card on a hand” would 
include lower-level routines such as “Find the list for the matching suit” 
and “Start a new suit list.’ Still lower we might find “Are card X and 
card Y of the same suit?’ List processing languages are especially de- 
signed to keep track of hierarchies of subroutines, so that the program’s 
course through subroutines within subroutines can always be unraveled. 

Recursions and push-down lists. List structures may have many 
levels of sublists, and routines for handling list structures will generally 
not know how many levels are involved in a particular structure. 
Routines must be built in a general way to handle any number of levels. 
One convenient way to achieve generality is to use a recursion. Suppose 
that we are writing a routine to perform a particular process on all the 
lists of a list structure. The process is first performed on the main list, 
then on every sublist named on the main list, then on every sublist 
named on a sublist, ete. Every symbol on the main list that names a 
sublist ean be treated as if it named a list structure; in fact, by the 
definition of a list structure, it does. But we are preparing a routine to 
perform a certain process on a list structure; so whenever we encounter 
a symbol that names a sublist, and therefore a list structure, we would 
like to execute the routine that we are now writing on this newly named 
structure. That is, the routine that we are writing wants to execute itself 
as a part of itself. Suppose, for example, that our routine is designed to 
substitute one symbol for another given symbol wherever the latter 
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occurs on a certain list structure. Our routine, which is diagrammed in 
Fig. 5-3, will examine each symbol on the main list. If the symbol 
matches the given symbol, then the substitution is made. If the symbol 
on the main list names a sublist, then the present routine is executed with 
that symbol as the name of the list structure involved. (It is assumed in 


Enter es Sl 





Initial set-up; store |_ input symbol, the 
replacement symbol, and the name of the list. 
Preset the current location on the 
current list a6 the list name. 








Obtain =e in current location 


| 


Does the obtained symbol match the input symbol? 


Substitute the 
replacement symbol 
































Does the obtained 
symbol name a sublist 
of this list structure? 


Execute subroutine S1 
with name of sublist 
as name of list structure 







Obtain link to next symbol on 
current list from current location. 
Store this link as new current location 





Is current list finished? 





Final clean-up: discard input symbol, replacement 
symbol, list name and current location 


Exit from subroutine S1 


Fig. 5-3. Flow diagram of subroutine S1, a ree ursion to substitute one symbol for 
another on a list structure, The double-lined box contains the entire flow diagram 
of routine Si, including a double-lined box that contains the entire flow diagram of 
routing S1, and #o on, ina “bakingspowder-can” rogrommlon, 
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this example that the given symbol does not name a sublist on the list 
structure.) When all the symbols on the main list have been treated in 
this way, the process is finished, since all the sublists will have been 
examined in the process above. A routine that uses itself as a subroutine, 
in this manner, is called a recursion. 

This recursion could not be programmed as a conventional computer 
subroutine. In conventional practice we would set aside three registers 
to store the three inputs—the input symbol, the replacement symbol, 
and the list name. Also, the value in the index register that links the 
subroutine to the main program either must be stored or must be left 
unaltered, so that the main program can be resumed at the proper place 
after the subroutine has finished its business. If the subroutine now 
executes itself, that linkage will be lost; it will be replaced by the new 
linkage to the place in the subroutine from which the subroutine was 
called. Also, the three inputs would replace the old inputs. In this par- 
ticular case, two of the inputs are the same (the input and replacement 
symbols), but the list name differs. The new list name would replace the 
old. Also, the new list name would be put in the storage register and 
the current location kept on the list, all record of the current location on 
the main list being thus destroyed, so that when the subroutine re- 
sumes after using itself, it will not know where to continue on the list. In 
short, its context has been destroyed. Whenever a subroutine is executed, 
a new context is set up and, in conventional practice, the new context 
completely destroys any previous context. A recursive routine must save 
its previous context before performing in a new context, so that when 
it finishes in the new context, it can revert to the middle of itself and 
proceed. Special provisions are made in list processing languages for 
saving contexts in recursions. 

The device used by recursive routines to save context is called a 
push-down list. For every storage cell containing either a parameter or an 
intermediate result, ie., every storage cell containing a part of the 
context, there is a push-down list on which the previous contents of the 
cell are saved. On a push-down list a new symbol is always added at the 
top; likewise, symbols are always removed from the top of the list, in 
the manner of a well for stacking plates in a cafeteria. When a new 
symbol is added at the top of the list, all the symbols beneath it are 
pushed down. As symbols are removed, they expose the symbols beneath 
them. Thus the symbols forming the new context of a recursive routine 
are stacked on top of the previous symbols. At the end of the routine, the 
present context is discarded, revealing the previous context. In the 
oxample in Fig. 5-8, the first and last blocks keep track of the context by 
manipulating push-down lists. The initial setup pushes down lists for 
storing the input symbol, the replacement symbol, and the name of the 
list and also pushes down a temporary storage cell that will hold the 
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current location on the list as the routine progresses. The final cleanup 
pops up these four push-down lists, leaving them exactly as they were 
when the routine was entered. 

Recursions are not a necessity in list programming, but they are an 
immense convenience. Conventional iterative procedures could be used, 
but a great deal of housekeeping (keeping track of all the lists, sublists, 
and so on) would have to be done explicitly, whereas with recursions the 
housekeeping is done automatically. Opportunities for recursions abound 
in artificially intelligent programs. Our bridge program will need a 
recursive routine for selecting a card to play. The selection will involve 
looking ahead to the possible plays from the other hands; i.e., what 
cards will the others then. select to play? These selections may be pre- 
dicted by using the ‘Select a card” routine in the context of the previous 
selections and the holdings of each other hand. Evaluating the conse- 
quences of a selection will also involve a consideration of future tricks; 
the same “Select a card” routine will be used again, and so a recursion 
naturally results. Each successive play that is considered involves a 
level in the recursion hierarchy of the ‘Select a card” routine. All 
game-playing programs have such “foresight” recursions. The big 
problem is when to stop recursing, sinee the machine seldom has time 
or capacity to look. ahead as far as it might. 

Particular languages. The most widely used list processing lan- 
guage is Ipt-v (Information Processing Language Five), the fifth version 
of a family of information processing languages designed by Newell, 
Shaw, Simon, and their collaborators (Newell et al., 1961). It includes 
over 150 basic processes for handling symbols, lists, and list structures. 
IpL programs are written as vertical lists of successive instructions, in the 
same way that machine-language codes are written. In contrast to 
conventional symbolic coding, all symbols used in Ipi programs must be 
in the form of a single letter followed by a relative number. The pro- 
grammer may not use his own private mnemonic symbols. 

The list processor designed by McCarthy, Minsky, and their collabo- 
rators at MIT is called Lisp and is very similar to lpi (McCarthy, 1960). 
It includes some novel technical features designed mainly to improve the 
speed of operation of the machine. It has fewer basic processes; the 
programmer must build up his routines from a much smaller set of basic 
operations. Mnemonic symbols are allowed, and the programs are 
written in a horizontal linear notation in the manner of symbolic logic 
expressions, with the use of parentheses. This notation is oriented toward 
mathematicians rather than computer programmers. 

Gelernter’s Fortran List Processing Language (I'LPL) is a set of 
routines within the ForrRAN system. Specifications for these routines 
have been published (Gelernter et al., 1960), If the programmer needs 
the resources of an algebraic compiler as well as a list processor, then 
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Fupu will be useful. However, producing the necessary routines from 
the published specifications is not a quick and easy task. An alternative 
is to embed algebraically compiled routines within the IpL or Lisp 
structures, since both have the facility for adding processes coded in 
machine language. 

Many other programming languages make use of the basic ideas of 
list memories. Yngve’s ComitT (1958), a compiler for handling linguistic 
material, uses a list memory. Most compilers of algebraic programs use 
lists to do their compilations. Perlis and Thornton (1960) have described 
a special type of list called the threaded list for use in compiler routines. 
Carr (1959) has also contributed to the development of general list 
processing. Fredkin (1960) has designed a more elaborate memory 
structure, called a trie memory, for data-retrieval problems. 

Use of list processing languages. The programming languages 
discussed here have been especially designed for complex, nonnumeric 
problems that have indeterminate storage requirements, that are 
naturally stated hierarchically and recursively, and that are subject 
to frequent changes. One type of problem for which the languages are 
especially suited is game playing. Of course, game-playing programs can 
also be written in machine language, but only with great difficulty and 
with important limitations. Chess programs written in machine language 
have generally looked ahead a fixed number of moves, whereas chess 
programs written in list processing languages can use recursions to look 
ahead a varying number of moves, depending on the circumstances. 
The chess program of Newell, Shaw, and Simon (1958), for example, 
evaluates the consequences of a move by looking ahead move after move 
until it reaches a position in which there is no possibility of one piece 
capturing another. The same kind of variable look-ahead could be in- 
corporated into a bridge program. List languages also provide convenient 
mechanisms for game-playing programs to use in, profiting from past 
experience. The past experience is often best expressed in terms of sets 
of associations, the associations among items being represented by their 
presence on a single list. 

Several programs that solve problems or prove theorems have been 
written in list languages. The first of these was the logic-theory machine 
of Newell, Shaw, and Simon, which produced proofs of theorems in 
logical calculus. Rather than using an all-inclusive algorithm for proving 
logic theorems, their program used a set of heuristics: rules of thumb 
which usually work in producing proofs but which have no guarantee 
of success. The heuristics were represented in the program by a list of 
possibilities: heuristics that worked were moved up on the list, while 
unsuecessful heuristics were demoted. Following the success of the logic- 
theorem prover, Gelernter and his associates (1960) used the FLex 
language to provide a computer with heuristics for proving theorems in 
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plane geometry. More recently, Tonge (1961) has written a program in 
Ipt for balancing a complex assembly line in a nearly optimum way, 
and Slagle (1961) has written a program in Lisp for doing formal 
integration. 

Problem-solving is very similar to game playing, in that a great many 
possible paths must be examined. The heuristics are intended to keep 
the program on or near the correct path and to give it ways of rejecting 
paths that will lead nowhere. The logic and geometry programs used a 
means-end analysis, working from the solution back to the premises. 
They sought a subproblem which, if proved, would allow them to prove 
the required results. In this way the problem was changed to finding a 
proof for the subproblem. Newell, Shaw, and Simon (1960), feeling that 
this analysis is common to a great many problems, have produced a 
general problem-solving program which uses a means-end analysis in a 
general framework. 

The heuristic procedures used in the logic and geometry programs are 
patterned after human behavior. Several attempts have now been made 
to produce programs that solve problems in the same way that human 
beings solve them, and other programs have been written to simulate 
other aspects of human behavior. Newell and Simon have adjusted 
parameters and subroutines in their logic program to simulate the be- 
havior of novices proving theorems in logic. The particular heuristics of 
a given subject are inferred from a protocol taken of his behavior in an 
experimental situation, and these rules are given to the computer. The 
computer program is then a model for the person’s behavior. (See 
Chap. 11.) 

List languages are also being used to program linguistic processes. 
Minsky and McCarthy are using Lisp for programming the advice taker 
(McCarthy, 1959). This program will accept declarative sentences in 
English as input and will abstract the relevant information from these 
sentences and remember it. This information, in the form of relations 
between objects, is then consulted to answer questions posed in English. 
A similar approach has been used by Lindsay in an Ipi program for 
language syntax analysis. 

List processing has turned out to be a very convenient way for pro- 
gramming Baseball, a program for answering questions posed in natural 
English about stored data (see Chap. 18). The linguistic part of Baseball 
is best programmed by treating the sentences as a list of words, each 
word having an attribute list containing its definition in terms of 
attribute-value pairs. The information-retrieval part of Baseball makes 
use of list structures for storing the data, which are baseball scores, and 
recursively searches the data for the answers to the question. By using 
lists both for the definitions of words and the organization of the data, a 
very general program can be written that is not geared to a specific 
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subject matter. A new context for answering questions can be provided 
by changing the dictionary lists and the data. 


Problems 


1. Write a Forrran program for each of the following processes. Assume that 
all necessary data are in storage; do not include input or output statements: 


. Find the largest of n numbers. 

. Rearrange a set of n numbers in the order of increasing size. 
Find the median of a set of numbers. 

. Calculate the percentile of each of a set of test scores. 
Calculate a ¢ test on two unpaired sets of n values. 
Calculate chi square for a fourfold contingency table. 


Ae as os 


(If your program has a DO loop that includes a GO TO or an IF statement 
the purpose of which is to start the next traversal of the loop, end the DO 
loop with the vacuous statement CONTINUE and assign this statement a 
number. This is the statement to which the GO TO or IF should refer in 
order to start the next loop traversal. CONTINUE adds no instructions 
to the compiled program; it merely helps the compiler to keep track of the 
flow of the program.) 

2. How can the illustrative program for standard scores be changed so that it 
produces exactly the same output but does not require a block of 1,000 
storage cells for T(I)? 

3. Rearrange the intercorrelation program so that R(I, J) is calculated directly 
from the sums and sums of square and cross products, rather than from the 
means, sigmas, and cross products. 

\. The program illustrating the calculation of intercorrelations does not use the 
fact that R(I, J) = R(J, I) to reduce the amount of storage required. Suggest 
some scheme for doing this. Write a correlation program that sets aside 
storage only for 144(M(M + 1)) correlations, when M is the maximum allow- 
able K. Use Forrran or some other algebraic compiler for which a manual 
is available. 

. Show by definition and example how you would set up a list structure for 
each of the following situations. In each case, indicate what other lists would 
probably be needed by the program: 


a, The organization chart of some business, laboratory, university, govern- 
ment, or similar organization 

b, A mathematical expression in ForrraAN notation 

c, An airline, railroad, or bus schedule 

d, A poker game 


e. A chess game 
J, Gin rummy 
g. The box score of a baseball game or some similar contest (see the sports 


page of a newspaper) 
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Digital codes 


il Nis basic ingredients of computer programming have now been 
presented, along with some typical examples, but many details have 
been ignored. To complete the exposition, it is necessary to examine more 
closely the manner in which computers use strings of digits to represent 
both numerical and nonnumerical information. 


6-1 Binary digits 


All computers use binary digits rather than decimal digits to represent 
stored information. A binary digit, or bit, has only two alternative values, 
() and 1, in contrast to the 10 possible values of a decimal digit, 0, 1, 2, 3, 
4, 5, 6, 7, 8, and 9. Binary digits are used for electronic simplicity. A 
hit can be represented by a circuit that is either conducting or not 
conducting current or by a magnetic element that is magnetized with 
one of two opposite polarities. One common binary device is a relay, 
which is an electromechanical switch that can be opened or closed auto- 
matically. Electronic equivalents of the relay form the basis of all digital 
computers. Because such components need to have only two stable states 

on and off—they can be made to operate very reliably, a property 
essential to good computer performance. 

Binary digits can be used to represent any kind of information. Infor- 
mation theory teaches that an item of information can be viewed as a 
selection of one out of a fixed number of alternatives. The two alternative 
values of a single binary digit, 0 and 1, can stand for any two alterna- 
tives, such as the right and left turns in a maze, or the yes and no 
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responses in an experiment on auditory sensitivity. The value of the 
digit represents the chosen alternative. A pair of binary digits can 
represent a selection of one out of four alternatives, such as the four 
compass directions, since there are four different ways in which the digit 
values can be combined: 00, 01, 10, and 11. A set of three binary digits 
can stand for 8 things, and, as Table 6-2 shows, a set of four digits can 
represent 16 things. In general, n binary digits can encode 2” different 
things. If the number of things to be encoded is not an exact power of 2, 
then the next higher power of 2 indicates how many binary digits are 
needed. Thus, 10 is not an exact power of 2, and the next higher power 
of 2 is 16, or 2; so four binary digits are needed to encode 10 things. 
Table 6-1 gives the decimal equivalents of successive powers of 2. If 
the alternatives to be coded have several attributes, it is often advan- 
tageous to code each attribute separately. For example, in encoding the 
52 playing cards of a bridge deck, separate digits could be used for the 
suit and the value of a card. Moreover, the value could be represented in 
the standard order from deuce through ace. Two binary digits would be 
needed to encode the suit (00 = clubs, 01 = diamonds, 10 = hearts, and 
11 = spades). Likewise, four binary digits would be needed to encode 
the value (0010 = deuce, 0011 = trey,..., 1011 = jack, 1100 = 
queen, 1101 = king, 1110 = ace). Similarly, a convenient code for the 
365 days in the year might use separate digits for the month and the 
day. Such a procedure is to be recommended even when digits are 
wasted, because of programming convenience. If different digits encode 
different attributes, the programmer can instruct the computer to ex- 
amine one attribute without disturbing the others. 

On rare occasions it is economical to consider codes with varying 
numbers of digits for the various alternatives. Suppose that a subject 
in an experiment can make either of two responses or can fail to respond, 
and suppose that the subject’s activity is coded once every second during 
the course of each experimental session. Then we can expect a great 
many more “no response”’ than responses. There are three alternatives 
—no response, response A, and response B—so that normally two 
binary digits would be used for each alternative. But if many more 
no-response codes are expected than either of the others, it would be 
economical to encode no response as 0 and to encode responses A and 
B as 10 and 11, respectively. With this code, the responses on successive 
seconds can be strung together without confusion. Thus the string of 17 


digits 


0 
1 


no 


0 0 
3 4 


ao 


Pb Oe 1. 2 10 O Ds 2 OD 
BF Be - Oo MONET Gn 40 ae 


represents 14 consecutive seconds, in which only three responses were 
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Table 6-1 Powers of 2+ 


2" n %26 
1 0 1.0 
2 1 065 
4 2 0.25 
8 3 0.125 
16 4 0.062 5 
32 5 0.031 25 
64 6 0.015 625 
128 7 0.007 812 5 
256 8 0.003 906 25 
512, 9 0.001 953 125 
1024 10 0.000 976 562 5 
2 048 11 0.000 488 281 25 
4 096 12 0.000 244 140 625 
8 192 13 0.000 122 070 312 5 


16 384 14 0.000 061 035 156 25 
32 768 15 0.000 030 517 578 125 


65 5386 16 0.000 015 258 789 062 5 

131 072 17 0.000 007 629 394 531 25 
262 144 18 0.000 003 814 697 265 625 
524 288 19 0.000 001 907 348 632 812 5 


1 048 576 20 0.000 000 953 674 316 406 25 

2 097 152 21 0.000 000 476 837 158 203 125 

4 194 304 22 0.000 000 238 418 579 101 562 5 
8 388 608 23 0.000 000 119 209 289 550 781 25 


16 777 216 24 0.000 000 059 604 644 775 390 625 

33 554 482 25 0.000 000 029 802 322 387 695 312 5 
67 108 864 26 0.000 000 014 901 161 193 847 656 25 
134 217 728 27 0.000 000 007 450 580 596 923 828 125 


268 435 456 28 0.000 000 003 725 290 298 461 914 062 
¢ ¢ 5 
536 870 912 29 0.000 000 001 862 645 149 230 957 031 25 
1 073 741 824 30 0.000 000 000 931 322 574 615 478 515 625 
2 147 483 648 31 0.000 000 000 465 661 287 307 739 257 812 5 


4 294 967 296 32 0.000 000 000 232 830 648 653 869 62 

8 906 25 
8 589 934 592 33 0.000 000 000 116 415 321 826 984 814 453 125 
7 1 79 869 184 34 0.000 000 000 058 207 660 913 467 407 226 562 5 
34 359 738 368 35 0.000 000 000 029 103 830 456 733 703 613 281 25 


68 719 476 736 36 0.000 000 000 014 551 915 228 366 85 

68 7 : 1 806 640 625 

187 438 472 37 0.000 000 000 007 275 957 614 183 425 903 320 312 5 
274 877 906 944 38 0.000 000 000 003 637 978 807 091 712 951 660 156 25 
D419 755 813 888 39 0.000 000 000 001 818 989 403 545 856 475 830 078 125 


| From Appendix, p. 139 of The Reference Manual for the IBM 709 Data Process- 
ing System, Used by permission. . 
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made. In scanning from left to right, the occurrence of a 0 means that 
there was no response, whereas the occurrence of a 1 means that this 
digit and the next digit are to be considered together and that the 
response was A if the pair is 10, B if 11. The pair 01 could not be used to 
encode another response in this scheme because the zero in the left digit 
in the pair would indicate a no response as the entire string is scanned 
from left to right. 

Because only one digit is used to encode the frequent no-response 
event, this special code would save considerable stored space. However, 
the program to manipulate the data would be complicated. The infor- 
mation can be decoded only if the starting place is known and only if the 
string is scanned from left to right. Scanning from right to left would, in 
general, lead to ambiguities. A more complicated code could be used if 
bidirectional scanning were necessary. We could use 0 for no response, 
101 for response A, and 111 for response B, but less storage would be 
saved. Which code is to be preferred would depend on the savings 
effected. Moreover, the programming difficulties far outweigh the 
storage savings, so that any code with more digits for the more frequent 
alternatives is likely to be uneconomical. 


6-2 Binary and decimal computers 


Computers based on binary digits can be designed in two fundamentally 
different ways. Binary computers use the binary number system com- 
pletely. All numbers are represented by strings of binary digits, and 
all operations, including arithmetic, are carried out in the binary system. 
This procedure gains speed and simplicity of internal computer oper- 
ations at the expense of translating inputs from decimal to binary and 
outputs from binary to decimal. Binary computers allow more flexibility 
in encoding nonnumerical information, but the programmer is forced to 
understand the binary number system. 

Decimal computers use binary digits only to encode decimal digits 
and then carry out all operations, both numerical and nonnumerical, 
with these synthesized digits. The decimal digits 0 to 9 might be encoded 
by the first 10 binary numbers in Table 6-2, but any other unique, one- 
to-one correspondence between the 10 decimal digits and sets of four or 
more binary digits would serve. Decimal computers sacrifice the sim- 
plicity and speed of the binary system for the familiarity of decimal 
numbers and compatibility with punched-card machines. Circuits for 
adding and multiplying the binary-coded decimal digits are more com- 
plex, and generally slower, than the corresponding circuits for the 
binary system. On the other hand, the programmer does not need to 
understand the binary system, since as far as he is concerned, the 
computer is dealing with decimal numbers, 
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Table 6-2 Binary-decimal integers 


Decimal Binary 
00 0000 
01 0001 
02 0010 
03 0011 
04 0100 
05 0101 
06 0110 
07 0111 
08 1000 
09 1001 
10 1010 
11 1011 
12 1100 
13 1101 
14 1110 
15 1111 


It might seem that the advantages of working in the familiar decimal 
number system would outweigh all other considerations. But the binary 
system Is very easy to comprehend, and, as we have seen, input-output 
subroutines can be used to take care of the binary-deciiaal translation 
problems. The programmer of a binary computer needs to know about 
binary numbers, but if he wishes, he may work entirely in the decimal 
system, leaving all the binary affairs to the computer’s library of utilit 
programs and subroutines. Only when debugging must the ene ae 
display any sophistication about the binary nature of the machine. In 
considering the pros and cons of various computers, whether a riachitie 
is binary or decimal has, it itself, very little weichb. 


6-3 The binary number system 


Strings of binary digits form binary numbers, just as strings of decimal 
digits form our familiar decimal numbers. The binary and decimal 
number systems are completely analogous, but the latter is so familiar 
that we seldom have occasion to ponder its properties, particularly the 
subtle use of position to determine the meaning of diviis The string of 
decimal digits 512 stands for 500 + 10 + 2. This can andes be wiitien as 
h x 10°? + 1 X 10'+ 2 X 10°, indicating that the digits are multi- 
pliers of successive powers of 10.! The same digits, in another arrange- 


\ ~matic i 
ie mathe matical notation the power of a number indicates how many times that 
Humber occurs in a product; for example, 10? = 10 * 10. Any number to the zero 


i ju unt W pl 1 1, N ative powers ndicate recipr ocals for 
ower equ als n ty; for example (° = € 1 
\ ’ B I 1 ’ 
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ment, 102.5, represent the number 1 X 10? +0 X 10'+2 X 10° +5 
x 10-1. The position of the digit, relative to the decimal point, de- 
termines the power of 10 that it multiplies, and the 0 must be included 
to keep the other digits in their places. 

In a binary number the digits are all either 0 or 1, and they multiply 
successive powers of 2. The string of binary digits 101 represents 
1x 2+0X2!'+1 X 2°, which equals the decimal number 5. In 
binary notation, the decimal number 102.5 becomes 1100110.1, that is, 
1X21 K2+O0X2+O0X24+1X 2 +1X2+0XK 
+1 x 27. (We may verify, decimally, that 102.5 = 64 +32+442 
+ 0.5.) Table 6-1 shows the decimal] equivalents of various powers of 2. 
It is helpful to remember that 2” is about 1,000, or about 10%, so that 3 
decimal digits are roughly equivalent to 10 binary digits, or 1 decimal 
digit to 3.3 binary digits. 

The simple regularities in the binary number system can be seen in 
Table 6-2, which shows the binary integers corresponding to the 
first 16 decimal integers 0 to 15. Counting is very simple in the binary 
system. The rightmost digit changes on every step. The next digit 
changes on every second step; the third digit on every fourth step; 
etc. One important detail is that the natural starting place is the number 
specified by a sequence of binary Os, that is, 0. The decimal custom of 
starting to count at 1 is difficult to change, but all sorts of programming 
complications arise unless binary counting starts at 0. Many pro- 
gramming errors can be traced to the simple problem of binary counting. 

Binary arithmetic. Doing arithmetic in the binary number sys- 
tem is only slightly more complicated than counting. The rules for 
binary arithmetic are completely analogous to those for decimal arith- 
metic, but we add and multiply decimal numbers so automatically that 
we are seldom aware of the rules. To add two numbers in the decimal 
system, we add the multipliers of corresponding powers of 10; thus, 
436 + 257 becomes (4 + 2) X 10? + (8 + 5) X 10' + (6 + 7) X 10° 
= 693. Whenever a sum of corresponding multipliers exceeds 9, one is 
added, or carried, to the next higher digit multiplier. The same rules 
apply in the binary system, but the addition table is much shorter: 
0+0=0,0+1=l1,andi+1= 0, with 1 to carry. (By extension, 
1+1+1 =1, with 1 to carry.) The reader may easily verify that 











1101 0 1 0 1D 3.1 1 1 1 0 
110 0 0 Oo. 1 1 1 0 1 
tA O20. 1 0 110. 0 1 1 0 Lt 


Subtraction is the inverse of addition and will not be discussed here. 
The decimal point in the decimal system has as its counterpart the 
binary point in the binary system, The generic name for this locator is 
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the reference point. This point specifies what power of the base (10 i 
the decimal system, 2 in the binary) each digit multiplies. Shiftin ig 
point corresponds to multiplying or dividing by the base n some ae 
of the base. Thus, in the decimal system, 1,025. is 100 times 10 28 yee 
1 .025 is one-tenth of 10.25. In the same way, in binary no bation 1101 
is four times 11.01, and 1.101 is half of 11.01. , 
Binary multiplication combines addition with shifting the bi 
point. The complete binary multiplication table isO X 0 = 0,0 X ioe. 
lex Le Consequently the structure of a multiplication problem * 
remarkably simple. As an example, a binary multiplication will be pl d 
alongside the equivalent decimal multiplication. —- 


25 11001 
x 13 x 1101 
75 11001 
25 00000 
325 11001 
11001 
101000101 


In the binary case, the partial products are either 0 or are equal to th 

multiplicand. The process of multiplication can be cea by Loe 
of a simple program that examines the digits of the multiplier from right 
to left and either adds or does not add the multiplicand. The flow 
diagram of this program is shown in Fig. 6-1. This program ig wired into 
the computer and is executed when multiplication is called for. Division 


is aki t ul plica 1 n, e; tr actions and will not 
Ss ntom ti t: oO requiring successive sub ) 
he discussed here. 


Notation. The base of a number system is the number whose 


a ta powers are represented by the successive digits. The base of 
me inary system is 2; the base of the decimal system is 10. When more 
‘han one base is in use, the base of a particular set of digits is often 
specified by a subscript, to avoid confusion. Thus 13 in the decimal 


system is written 13; and is equival ; 
t ils 
11001012; 10.2510 = 1010.01>. q ent to 1101,. Similarly, 10119 = 


6-4 Octal numbers 


ie one deind defect of binary digits is that so many of them are needed 
0 encode a moderately large decimal number. For example, the bin 

equivalent of the decimal number 9,572 is 10,010,101 100 100. S he 
“i of abbreviation is needed in order to denote the binady hen 
10 accepted abbreviation is to group the bits in threes, starting at Siig 
inary point, and to call these groups by their decimal equivalents. In 
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our example, we would say or write 22544, since 0102 = 210, 1012 = 5ioy 
etc. Note that we use only the eight digits 0 to 7 in this abbreviation. 
Thus 000 = 0, 001 = 1, 010 = 2, 011 = 3, 100 = 4, 101 = 5, 110 = 6, 
111 = 7. In fact we are using the octal number system, in which various 
digit positions in a number denote various powers of 8. Thus the octal 
number 22544 is2 X 8!+2x*8+5xX8+4xX8'+4xX 8°. Clearly, 
the octal number system has the property that one octal digit can be 
encoded by exactly three binary digits, since that is how we happened 
onto the octal system in the first place. The reference point—the octal 


START 


Preset product to zero 


















Get right-most digit of multiplier 






Is multiplier digit 1? 





Add multiplicand to product 
Shift multiplicand left one digit position 














Get next digit to left, if any, in multiplier 


STOP 


Fig. 6-1. Flow diagram of binary multiplication. 


point in this system—is important as the referent in coding binary to 
octal. Thus the binary number 1011.01 becomes the octal number 13.2, 
since the groups of three digits are counted left and right from the point. 

Octal numbers are indispensable in dealing with a binary computer. 
For one thing, the rows of flashing lights on the computer console are 
rows of binary numbers. The most economical way to interpret these 
lights is to perceive them as octal numbers, and the lights are generally 
grouped in threes to facilitate this perception. With a little experience, 
converting from binary to octal or from octal to binary becomes auto- 


matic. \ . 
The octal system is not a universal shorthand for binary numbers. 
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At least one computer prefers a hexadecimal code. In the hexadecimal 
number system, each digit represents a power of 16, and each digit may 
have one of 16 possible values. The numerals are used for 10 of these 
values, and special symbols are used for the remaining 6. 1315 = 191 
= 233 = 10011.. 


6-5 Computer words and word formats 


Computers combine digits into larger units called words. For example, 
a decimal computer might combine 10 decimal digits into one word 
while a binary computer might have words of 36 binary digits each. The 
number of digits in a computer word is called the word length; it is fixed 
for each computer but generally differs from one computer to the next. 
Our hypothetical computer has a word length of 36 bits. Each storage 
register in the computer can hold one word, and the accumulator also 
has the capacity of one word. The computer’s instructions, such as 
LOAD FROM, STORE IN, ADD CONTENT OF, etc., refer to one 
word. Thus, the instruction LOAD FROM 200 copies the values of 
the 36 digits in register 200 into the corresponding 36 digit positions of 
the accumulator, while the instruction STORE IN 200 does the reverse. 

Numbers. In numerical calculations, each number is usually repre- 
sented by the digits in one computer word. Both positive and negative 
numbers must be accommodated. On decimal machines, a special binary 
digit is usually appended to each word, to code the sign; so a typical 
word length on a decimal machine might be 10 decimal digits plus 1 
binary digit. A negative number may be represented by a negative sign 
(i.e., a 1 in the sign bit) and the absolute value of the number in the 
numerical digits. In this representation, +14 and —14 would differ only 
in the sign bit. For every positive number there is a corresponding nega- 
live number, obtained by changing the value of the sign bit. Unfortu- 
nately, this principle also applies to the number 0. Both +0 and —0 are 
possible. Sometimes, as on our hypothetical computer, the arithmetic 
circuits are wired so that only +0 or only —0 can occur as the result of 
wn arithmetic operation. In general, though, the programmer must be 
prepared for both Os. 

Many machines use complements to represent negative numbers, to 
simplify the circuitry. On a 10-digit decimal machine, the so-called 
lens complement of 14 is 10,000,000,000 — 14, or —9,999,999,986. The 
sign is minus to indicate that this is a complement rather than a huge 
positive number. Binary computers use equivalent binary complements. 
The use of complements simplifies the arithmetic, because subtraction 
may be replaced by the addition of complements. Complements will not 
he discussed here, for while the programmer must know a little about 
his computer's system for coding negative numbers, he can leave most 
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of the problems to the in-out library subroutines. The details of comple- 
ment arithmetic can be found in McCormaik (1959), Richards (1955), 
ny other excellent sources. 
gece fe numbers, the position of the decimal or binary point must 
also be specified. Generally, it is specified to be at the left of the nu- 
merical digits, so that all the numbers are treated as fractions. The coded 
form of these numbers, which are called fixed-point numbers, is depicted 
in Fig. 6-2a. To perform calculations with Jarger numbers, the pro- 
grammer must shift the decimal point in all his numbers to reduce them 





Numerical Value 


Sign 


(a) Fixed point numbers 


Value of Scale Factor Value of Fraction 


cal 


cS Sign of Fraction 
Sign of Scale Factor 








(b) Floating point number 
Operation (unused) index Address 
1 6 14 2 12 
MS toameedl 


(c 


) Instruction 


char. char. char. 


char. char. char. 


(d) Alphanumeric characters 


Fig. 6-2. Word formats for a hypothetical 36-bit binary computer. (a) Fixed-point 
number; (b) floating-point number; (c) instruction; (d) six alphanumeric characters. 
The number of bits are shown in each section. 


to fractions. For example, the numbers 512 and 329 might be put into 
a decimal computer as 0.512 and 0.329. The decimal point in the answer 
will then have to be changed back by some amount depending on the 
multiplications and divisions in the calculation. For additions and sub- 
tractions, the decimal point can be considered to be any place at all, 
so long as it is in the same place for all numbers in the calculation. In 


particular, with integers it is sometimes preferable to consider the — 


decimal point to be at the right end of the word, This would be the case 
with counters or tallies, where the only operation is adding unity. In 
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short, the decimal place is movable conceptually, but the programmer 
must keep track of it. 

The decimal point can become very “pesky” in automatic calculation. 
Slide rules completely disregard the decimal point, yielding only the 
significant digits of the answer, for example, 5 1 2; it is up to the user to 
keep track of the point, so that he will know whether the answer is 51.2 
or 0.512 or has some other scale factor. Desk calculators likewise operate 
essentially in ignorance of the decimal point, although there are rules 
and “gadgets” that can be used to help keep the point in place. In 
essence, desk calculators act as though the decimal point were fixed at 
the right-hand end of the keyboard and carriage, so that all numbers 
are integers. To work with fractions, it is necessary to multiply then by 
a power of 10, or scale factor, so that the answer can be divided by it or 
by some suitable combination of the scale factors of the various numbers 
entering the calculation. The rules and gadgets are memory aids. Thus, 
0.0512 X 0.004 = (512 K 10-*) X (4 X 10-8) = 512 & 4 & 10-4 & 10-3 
= 2048 X 1077 = 0.0002048. Likewise, 0.0512 + 0.004 = (512 x 10-4) 
+ (40 X 10~*) = 552 X 10-4 = 0.0552. Of course, in simple problems 
the position of the decimal point is obvious, and the scale factors are 
never considered consciously. As the calculations become lengthy, 
though, the scale factors become a problem. Often the same scale factor 
will do for all the numbers in a calculation, in which case the decimal 
point can be “set” on the keyboard. However, if it becomes clear, part- 
way through the calculation, that a different scale factor is needed, it 
can be changed. 

Scale factors are real bugaboos in digital computers, because the 
calculation is fully automatic, and so there is no opportunity to adjust 
the scale factor during the course of the problem. A set of rules and 
procedures can be set down that will help the programmer to “scale the 
problem,” and many programming courses include a discussion of this 
lopic. Sealing is a burdensome task, but we shall not discuss it further, 
hecause it can usually be avoided. 

Since precise rules can be specified for keeping track of scale factors, 
it follows that computers could do the scaling automatically. Most 
vomputers have this capability, which is achieved either by means of 
tility subroutines or by means of special built-in operations. The 
lochnique is to represent each number as both a fraction and a scale 
luctor. Some of the digits in the word are set aside to code the scale; the 
rest are used for the fraction, as shown in Fig. 6-2b. Thus 512 is repre- 
vented by the pair of numbers 512, 3, meaning 0.512 X 10%. Negative 
scale factors are also possible: 0.00512 becomes 512, —2, that is, 0.512 
* 10~*. Because the scale factor essentially represents the location of 
the reference point and has a wide range of possible values, the reference 
point is called floating rather than fixed, If 8 bits are used to code a 
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binary scale factor and 1 of these is for the sign of the scale factor, the 
range of possible scale factors is from — 128 to +128. It follows alist ue 
upper limit of floating point numbers is 2", which is roughly 10*, a 
number so large that it is encountered mainly by astronomers and 
atomic physicists. 

Floating-point operations relieve the programmer from concern about 
the reference point. The price that must be paid is less precision in the 
coded number (for example, 27 bits rather than 35) and slower speed. 
The slowness results from the more complicated rules for addition and 
multiplication, since both the scale factors and fractions must be com- 
bined according to different rules. This price is generally regarded as 
low, perhaps even a downright bargain. 

One small issue remains. What happens when the result of a caleu- 
lation is too large for the computer? The sum of two fixed-point fractions 
can easily exceed the upper limit of unity, and there are even some rare 
occasions when a floating-point calculation produces a number with too 
large a scale factor. In both cases, the condition is called overflow and 
is generally sensed by the computer. A floating-point calculation = 
also produce a number too small for the machine, for example, 5 X 10~*°. 
This condition is called underflow and is also sensed by the computer. 
Sometimes the computer can be set to stop whenever an overflow or 
underflow occurs; sometimes the programmer can test the condition as 
a part of his program, by means of a conditional instruction such as 
JUMP IF OVERFLOW, and can thus care for the situation if he has 
foreseen its potential occurrence. Generally, the results of a calculation 
are rendered meaningless when overflow or underflow occurs, unless the 
situation is immediately corrected in the program. Por example, on 
most machines, if the overflow of a fixed-point addition is ignored and 
the result is stored, the stored number will consist of only the fractional 
part of the sum; the integer (which caused the overflow) will be lost. 

(Thus, 0.7 + 0.5 = 0.2.) 

Overflow can sometimes be helpful. One scheme for generating ran- 
dom numbers (see Chap. 9) is based on the fact that, if two random 
numbers between 0 and 1 are added modulo 1 (meaning that only the 
fractional part of the result is to be retained), this resulting number is 
statistically independent of either of its parents. Moreover, it can be 
added to one of its parents, modulo 1, to produce yet another inde- 
pendent number, and thus a sequence of random numbers can be 
generated. The sum modulo 1 is accomplished neatly by using fixed- 
point addition and ignoring overflow; so the subroutine to generate 
random numbers in this fashion is remarkably simple. 


Instructions. To encode instructions, the computer word is divided 


into three or more distinet parts, each of which holds a necessary code 
value, In our hypothetical computer, codes are needed for the operation, 
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the address, and the index register. The operation code is a set of digits 
specifying the particular operation to be performed. This code occupies 
as many digits as necessary for the number of different operations in the 
computer’s repertoire. A computer with 50 operations would need 6 
bits, or two decimal digits, to encode the operations. Likewise the 
address code specifies the storage register involved in the instruction. 
A computer with 4,096 storage registers would number them consecu- 
tively from 0 to 4095, thus needing 12 bits, or four decimal digits, to 
encode the address. Finally, a third built-in code is needed to specify 
which index register is involved. Since the code must include a value for 
“no index register,” a computer would need 2 bits to encode two index 
registers. The assignment of code numbers to operations and index 
registers, and the assignment of addresses to storage registers is fixed by 
the computer wiring and cannot be altered. The complete instruction 
format is illustrated in Fig. 6-2c. Note that some of the digits are not 
used at all. Special uses for the remaining digits are sometimes found. 
Alphanumeric characters. It is often necessary to store alpha- 
betic information in the computer. Generally, a word format is used 
that allows compact storage of text composed of alphanumeric characters 
the letters of the alphabet, the numerals, some special symbols, and 
‘blank.’ There are 26 letters and 10 numerals, so that if 6 bits are used 
for each character—allowing 64 characters—many special symbols can 
be accommodated. Six of these 6-bit characters can be stored in a 36-bit 
word, as shown in Fig. 6-2d. Generally, the particular pairings of code 
numbers with characters is completely arbitrary and can be chosen for 
convenience. It might be profitable, for example, to use the first 10 code 
numbers, 0000002 to 0010012 (00s to 11s), for the numerals O10 to Qyp. 
Often the design of the input and output mechanisms dictates the use 
of a particular code. As stated in Chap. 2, the electromechanical input 
ind output machines are generally more rigid about digital codes than 
the computer. 
Other word formats. The programmer is free to use the digits in 
. word in any way that he pleases to encode his data. If, for example, 
his data are encoded in one long string of digits, the string must be 
broken into one-word segments, which can then be stored in successive 
registers in memory. If the programmer uses this or some other special 
word format that is not built into the computer, he will have to write 
subroutines and programs to manipulate the digits according to his 
format, thus in essence providing the computer with the rules for his 
vode, These programs use instructions that will be introduced in Chap. 7. 
Variable word length. One popular computer, the IBM 1620, has 
in unusual design. It has no fixed registers, and its word length can be 
whatever the programmer wants, The computer is a medium-speed, 
decimal machine, and each decimal digit in the memory can be addressed 
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individually. All operations within the machine take place digit by digit. 
The computer can handle as a unit any digits that are contiguous in 
memory. A group of digits that is to be treated as a unit is called a field 
and is separated from other fields by a special flag bit in the binary code 
for the leftmost digit in the field. The address of the field is the address 
of the rightmost digit in the field—we are thinking of the digits in 
memory as if they were all stretched out in one long row and numbered 
from left to right. Operationally a field is equivalent to a computer 
word, but the field length is arbitrary on the IBM 1620, whereas the 
word length in a normal computer is fixed at the capacity of a register. 


Problems 
1. Convert the following decimal integers to binary: 
1710 3110 6910 5410 12010 
2. Convert the following binary integers to decimal and octal: 
1101, 10010. 11011; 100100: 110001. 
3. Add the following pairs of binary numbers. Verify your answer decimally: 


1101.0 + 10010.0 1101.1 + 1101.1 
11010.0 + 10010.0 11.011 + 1.1 


4. Multiply the following pairs of binary numbers. Verify your answer decimally. 


1101.0 x 101.0 100.1 x 10.1 
1010.0 x 1010.0 111.11 X 10.11 
1101.1 x = 11.01 


5. Write out the octal addition table and multiplication table. 


6. The Gray code is a special binary code for numbers. The code values for any 
two successive numbers differ from each other by only one digit. (The Gray 
code is useful in digitally encoding the angular position of a continuous 
rotary control, since rotating the control changes only one digit at a time.) 
A 3-bit Gray code for the numbers 0 to 7 is as follows: 


0 000 
1 001 
2 O11 
3 010 
4 110 
5 M1 
6 101 
7 100 
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Use the same principle to write a 4-bit Gray code for the numbers 0 to 15. 


7. Assign 38-bit binary-code values to four events so that each code value 
differs from each of the other three in at least two digits. How many different 
sets of four 3-bit binary numbers have this property? 
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Programming for information processing 


HE computer instructions that have so far been introduced have, 

with one exception, dealt with the computer word as a unit. 
Additional instructions are needed to process parts of words and other 
coded information in which the computer word is not the natural unit. 
An important special case is the computer instruction itself. Some of 
the digits in an instruction word represent the operation to be performed, 
other digits represent the address of the operand, and still others represent 
the index register involved. It will often be convenient to manipulate 
the three sections of the instruction word separately. In this chapter, 
instructions will be presented that are designed to manipulate infor- 
mation either bit by bit or in units smaller than a full computer word. 
The in-out instructions will also be mentioned, as will possibilities for 
external control of the program and special features found on some of 
the more elaborate computers. The discussion will be confined to binary 
machines; the same principles apply to decimal machines. 


7-1 Part-word operations 


The word format for an instruction in our hypothetical computer in- 
cludes parts specifying the operation to be performed, the address of 
the data, and the index register, if any, as shown in Fig. 6-2c. The 
ability to manipulate each part separately is a great convenience. In 
particular, an instruction that we have already used to great advantage 
in the bookkeeping example in Sec. 3-5 is STORE ADDRESS IN Y. 
This instruction replaces the address part of the word in register Y by 
118 
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the address part of the word in the accumulator. The instruction 
STORE ADDRESS IN Y is preferred to STORE IN Y in most self- 
modifying programs, because it guards against the programming bug of 
leaving the wrong information in the other parts of the accumulator. 
Also we may want to store the same address in several instructions that 
have different operation codes. If the accumulator contains a word with 
the required address, we may use a succession of store address in- 
structions rather than having to load the accumulator every time with 
a word having the complete instruction format. 

Less frequently it is helpful to have instructions that store the other 
parts of the instruction word. Thus, for our hypothetical computer, 
we may define the instruction STORE OPERATION IN Y, which 
replaces the operation digits in location Y by the corresponding digits in 
the accumulator. A similar definition applies to the instruction STORE 
INDEX IN Y. There is no engineering difficulty in providing such 
instructions; a part of a word can be manipulated as easily as a complete 
word. There is, however, an advantage in keeping the total number of 
instructions small. The fewer instructions, the less circuitry is needed 
in the computer. Thus, on some computers, the parts of an instruction 
may be manipulated only by the more general instructions introduced 
in the next section. 


Illustrative example: squaring routine. Suppose that N numbers are 
located in successive registers beginning at register X. Each is to be squared 
and the result replaced in the original register. Suppose further that this must 
be done without benefit of index registers. A program to perform this operation 
is shown in Table 7-1. First the addresses of the three instructions in the squaring 
loop are preset by getting the initial address into the accumulator and entering 
the loop at the address modification point. On every cycle through the loop, 
three instructions with different operations but the same address must have the 
address modified. The advantage of the store address instruction is evident. 
Note that the numbers are processed in reverse order, starting with the number 
in X + N — 1, and ending with the number in X. 


7-2 Logical operations and packing 


When items of nonnumerical information have each been coded in a few 
hits, several items can be stored in one register. For example, alpha- 
numeric symbols (letters and numerals) are often coded in 6 bits; conse- 
quently six characters can be stored in a 36-bit register, as shown in 
l'ig, 6-2d. Numerical information can be stored compactly, several 
numbers in each register, if each of the numbers is small. Whenever 
several items of information are stored in one register, the information 
is said to be packed, Operations are required for packing and unpacking 
these items, In general, we want to be able to extract certain bits from a 
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Table 7-1 Squaring routine 


i 





Location | Content of register Comments 
Program 
START LOAD FROM TEST 
ADD CONTENT OF PRESET | Start with 
X+N-1 
JUMP TO IN Enter loop to preset 
LD LOAD FROM xX Square the number 
MP MULTIPLY BY CON- 
TENT OF xX 
st STORE IN x 
LOAD FROM LD Modify addresses 
SUBTRACT CONTENT OF ONE 
IN STORE ADDRESS IN LD 
STORE ADDRESS IN MP 
STORE ADDRESS IN ST 
SUBTRACT CONTENT OF TEST Check if done 
JUMP IF PLUS TO LD 
JUMP TO NEXT Go if done 
Constants 
ONE +1 
PRESET N-1 
TEST LOAD FROM x 
Data 
xX Block of N registers 


a 


computer word and, after processing, to store the same bits of the word, 
without disturbing the other items in the register. The specific bits to be 
extracted, processed, and stored are specified by a mask: a binary word 
containing Is in those bit positions containing the information that we 
wish to manipulate and Os in those bit positions containing the infor- 
mation that we wish to leave alone. Then we can use the following in- 
structions: 


MASK FROM Y. The content of the accumulator is compared with 
the content of register Y, bit by bit. Wherever the bit in the ac- 
cumulator is 0, it remains 0. Wherever the bit in the accumulator is |, 
it is replaced by the value of the corresponding bit in register Xi 

FILL IN Y. The content of the accumulator is compared, bit by bit, 
with the content of register Y. Wherever the bit in the accumulator 
is 0, the corresponding bit in register Y is unchanged, Wherever the 
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bit in the accumulator is 1, the corresponding bit in register Y is made 
al. 

COMPLEMENT. Each bit in the accumulator is reversed. Each 0 
becomes a 1, and each 1 becomes an 0. 


To extract an item of packed information from a particular part of 
register Y, we must load a mask into the accumulator and execute the 
instruction MASK FROM Y. For example, if we wish to obtain the 
last 6 bits in register Y, supposing them to be a particular alphanumeric 
character, we would load into the accumulator a mask consisting of Is 
in the last 6 bits and 0s elsewhere. Then the instruction MASK FROM 
Y will leave the last 6 bits from register Y in the accumulator; the other 
30 bits in the accumulator will still be 0. Note that the masking operation 
does not require contiguous bits; any set of bits could be masked, since 
cach bit position is examined independently. 

Now consider the reverse process. Suppose that the last 6 bits in the 
accumulator represent an alphanumeric character that we wish to place 
in the last 6 bits in register Y. Suppose that the accumulator is all 0 
except for its last 6 bits. If we are also assured that the last 6 bits in 
register Y are all Os, then we may merely execute the instruction FILL 
IN Y. However, if we cannot be sure that the last 6 bits of register Y are 
Os, then our first problem is to make them Os. For this purpose we need 
u mask that is the complement of the mask we used in the earlier 
operations, namely, a mask with all 1s except for the last 6 bits, which are 
Os. Normally we would load such a mask into the accumulator, but in 
order to illustrate the use of the operation COMPLEMENT, we shall 
load our original mask into the accumulator and then execute the in- 
struction COMPLEMENT. The effect of this instruction is to switch 
all the bits in the accumulator, thus providing us with our required mask. 
Ilaving the mask in the accumulator, we then MASK FROM Y and 
STORE IN Y. This pair of instructions will bring the first 30 bits of 
register Y into the accumulator, leaving the last six bits in the ac- 
cumulator Os. By then storing the entire word in register Y we have in 
effect cleaned out the 6 bits in question. Then we may bring into the 
accumulator the character that we wish to deposit in register Y and 
proceed to FILL IN Y, as shown in Table 7-2. 

Matching and testing. Mathematical computer programs typi- 
cally decide whether or not to continue a particular calculation according 
to which of two numbers is larger. The instructions JUMP IF MINUS 
and JUMP Il’ PLUS seem the most natural under those circumstances. 
In dealing with nonnumerical information, however, we are generally 
looking for a symbol or code that matches some specified symbol, and 
our decision will depend on the occurrence or nonoccurrence of an exact 
match, lor this purpose we need an instruction to match two items, 
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Table 7-2 Depositing a character 





Location Content of register Comments 

Program 
LOAD FROM 6MASK 1s in rightmost 6 bits 
COMPLEMENT 1s in leftmost 30 bits 
MASK FROM Y Pick up first 30 bits in Y 
STORE IN Y Last 6 bits 0 
LOAD FROM CHAR 
FILL IN ¥ 


(Rest of program) 


Constants 
6MASK 000 000 000 077 (Octal) 
CHAR 000 000 000 062 (Octal) 
Y 234 665 255 163 (Octal) 





MATCH WITH Y and also the conditional instruction JUMP IF 
ZERO TO Y: 


MATCH WITH Y. The content of the accumulator is compared, bit 
by bit, with the content of register Y. Wherever the 2 bits are the same 
the bit in the accumulator is set to 0; wherever the bits are different, 
the bit in the accumulator is set to 1. 

JUMP IF ZERO TO Y. If each bit in the accumulator contains a 
zero, the program counter is set to Y, so that the computer takes its 
next instruction from register Y. Otherwise the program counter is 
advanced normally. 


To compare an item of information in the accumulator with the stored 
item in register Y, we have merely to execute the instruction MATCH 
WITH Y followed by JUMP IF ZERO TO . The jump 
will occur if the two items match and will not occur if they do not match. 

The operations MASK, FILL, and MATCH are generally termed 
logical operations because of their correspondence with concepts from 
mathematical logic. The operation MASK FROM Y is often called 
and, since the bit-by-bit operation is equivalent to logical and. That is, 
each bit in the accumulator is made 1 if and only if both the bit in the 
accumulator originally and the corresponding bit in the storage register 
Y are 1. If either or both bits are 0, then the result is 0, FILL IN Y 
corresponds to logical or. For each bit position the result is | if the corre- 
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sponding bit in either the accumulator or the Y register, or both, is 1. 
The bit position is 0 only if both are0. MATCH WITH Y corresponds 
with the logical operation exclusive or. For each bit position, the result is 
1 if either the bit in the accumulator or the bit in the storage register 1s 
1 but is 0 if both are 1 or if both are 0. The terms and, or, and exclusive 
or are often used by computer engineers in preference to the more 
descriptive MASK, FILL, and MATCH. 

Computers differ in the number and kind of logical operations that 
they provide. Some computers have only the mask operation. By suita- 
bly manipulating masks, the fill operation can be simulated by an add 
operation. If the bits to be filled are all Os and the filling word has Os 
except for those bits which are to be filled, then no bit position has a 1 
both in the filling word and in the filled word. In this case binary addition 
is equivalent to logical or since 0 + 0 = 0,0 +1 = 1,1 +0 = 1. Only 
in the case of 1 + 1 does binary addition differ from logical or. Similarly, 
matching can be done by a subtraction process. The two coded items of 
information can be treated as numbers; subtracting one from the other 
will yield 0 if and only if the numbers are identical. Using this method, 
however, requires careful attention to the particular way in which 
positive and negative numbers are coded in the computer being used; 
the user must also beware of plus and minus 0. If the computer is to be 
used extensively for nonnumeric calculations involving items of infor- 
mation packed in computer words, then a large variety of logical oper- 
ations, such as the set we have provided for our hypothetical computer, 
is very helpful. 


Illustrative example: matching. Information is stored in N registers 
beginning at register X. The last 6 bits of each register contain an item of infor- 
mation that is to be examined. If that information matches the item stored 
in the register labeled COMP, then it is to be replaced by the item stored in the 
register named NEW. If the item from register X does not match COMP, 
then there is to be no change. 

The program is shown in Table 7-3. The first instruction presets index register 
A, The next two instructions obtain the 6 relevant bits from the storage register. 
Note that index register A must start with the number N — 1, since the stored 
information is in registers X through X + N — 1. Having obtained the infor- 
mation from storage, we compare it by a match operation, and if the match 
occurs, we make the replacement by jumping to the part of the program starting 
ut FLX, If the match fails, we merely index for the next storage register. At 
VIX we first obtain the complement mask and then obtain all the information 
except the last 6 bits from the storage register. We then store the result so that 
the last 6 bits are now Os. Then we fill these bits from NEW, which we know 
line Os in the first 30 bits. We then go to INX, where we index for the next 
traversal of the loop. 
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Table 7-3 Program to replace a certain item by another 


SHIFT LEFT N. The digits in the accumulator are shifted left N bit 
positions. The digit in bit position 36 is shifted to bit position 36-N, 



















































Location Content of register Comments etc. Bits shifted past position 1 are lost. Bit positions made empty by 
Promina the shift are filled with 0s. 
START LOAD A FROM NUMBER | Preset A to N —1 SHIFT RIGHT N. The digits in the accumulator are shifted right N 
LOOP LOAD FROM MASK Test 6 iita-are te bit positions. The information in digit position 1 is placed in bit 
MASK FROM X:A Get last 6 bits from position N + 1, etc. Digits shifted past bit position 36 are lost; bit 
block positions made empty by the shift are filled with Os. 
MATCH WITH COMP Does it match? 
JUMP IF ZERO TO FIX Go if match 
INX INDEX A FROM MONE | No match; leave as is 
JUMP IF A PLUS TO LOOP Count down in A ar Aa 4 
JUMP TO NEXT On to next program 
FIX LOAD FROM MASK Last 6 bits 1s O10T Le. ke ele 400 1 0-2, 1200 0 
COMPLEMENT First 30 bits 1s 
MASK FROM X:A Clean out last 6 bits + (a) 
STORE IN X:A 
LOAD FROM NEW Replace last 6 bits 
FILL IN X:A 
JUMP TO INX Return to index loop ee Oe ee a 
Constants \ a 
NUMBER | N—1 O 0" :0 OL. OCF (Ode ie Seas 
MASK 000 000 000 077 (Octal) (b) 
MONE —1 
COMP 000 000 000 021 Comparison character 
NEW 000 000 000 032 Replacement character 
Data 
x Block of N storage a er Ge 4 


(c) 


Fig. 7-1. Action of shift and rotate instructions. (a2) SHIFT LEFT 3; (b) SHIFT 
RIGHT 3; (c) ROTATE LEFT 3. 


7-3 Shift operations 


To complete the computer’s facility for handling bits of information, we 
must provide the ability to move bits from one bit position to another 
within a register. For example, we might wish to move an item of 
information from the last 6 bits of one register to the first 6 bits of 
another register. To do this, we must somehow shift the information from 
the last 6 bits in a word to the first 6 bits of that word. This is done in 
the accumulator, using shift instructions. For our hypothetical computer 
we shall define three instructions. In the definitions we shall adopt the 
convention of numbering the bit positions in a computer word from 1 to 


ROTATE LEFT N. The digits in the accumulator are shifted left N 
bit positions, as in SHIFT LEFT N. However, the digits shifted left 
past bit position 1 are brought around and entered in the bit positions 
at the right end of the accumulator emptied by the shift. 


The effect of the instructions is shown in Fig. 7-1. 
‘To move the information from the last 6 bits in a register named 


36, with 1 on the left and 36 on the right. (These conventions differ from 
one machine to the next.) 





LAST to the first 6 bits in a register named PACK we would execute the 
following three instructions ; 
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LOAD FROM LAST 
SHIFT LEFT 30 
FILL IN PACK 


This assumes that the first 6 bits in PACK initially were 0. Note, how- 
ever, that all of the bits in register LAST except the last 6 bits are 
irrelevant, since they are shifted out of the accumulator and lost. Note 
also that in the shift and rotate instructions the address part of the 
instruction does not contain the address of a storage register but is used 
instead to indicate the number of bit positions that the shift should 
traverse. No storage register is involved in any of the shift and rotate 
instructions; the operation is concerned solely with the accumulator. 


Illustrative example: unpacking alphanumeric characters. Suppose 
that a register named PACK contains six alphanumeric characters each encoded 
by six bits (as in Fig. 6-2d). We wish to unpack this information, storing each 
character in a separate register. Each character is to occupy the last six bit 
positions of its register. The six registers that are to receive the six characters 
start at a register named LIST. The program shown in Table 7-4 will perform 


Table 7-4 Unpacking routine 





Location | Content of register Comments 

Program 
LOAD A FROM ZERO Preset A for LIST location 
LOAD B FROM FIVE Preset B for loop count 

LOOP LOAD FROM PACK 
ROTATE LEFT 6 Position next character 
STORE IN PACK Save all bits, in rotated 
position 

MASK FROM MASK Clear all but last 6 bits 
STORE IN LIST:A 
INDEX A FROM ONE 
INDEX B FROM MONE 


JUMP IF B PLUS TO LOOP 
(Rest of program) 


Constants 
MASK 000 000 000 077 Octal 
ZERO 0 
FIVE +5 
ONE +1 
MONE —1 
Data 
PACK 476 270 233 046 Octal 


LIST 0 Block of 6 registers 
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the unpacking process. First index register A is set so that the first character 
will go in the first register in the six-register block. Then index register B is 
preset to 5 so that the loop can be counted six times, starting with 5 and ending 
with 0. The loop then starts at the register named LOOP, and the packed 
information is brought into the accumulator, rotated left 6 bits, and returned 
to PACK. Of course it also remains in the accumulator. At this point on the 
first cycle through the loop, the first 6 bits from PACK are now correctly posi- 
tioned in the last 6 bits of the accumulator. We then mask the 6 bits in the 
accumulator, clearing out the remainder of the accumulator, and store the 
result in the first list cell. Then both index registers are advanced, and the loop 
is tested for completion. On the second pass through the loop we note that the 
second alphanumeric character, character 62 in PACK, which originally occu- 
pied bit positions 7 to 12, now occupies bit positions 1 to 6 as a result of the 
rotation and restorage in the first cycle. Consequently the second rotation will 
bring those bits into the last six bits of the accumulator on our second cycle; 
they are then in position for masking and storing. At the end of six traversals 
of the loop the work is done. Note that the routine leaves the information in 
PACK exactly as it was found. 


Many machines have additional shift instructions involving the 
register that serves as an extension of the accumulator. This register, 
called the B register, the MQ register, the quotient register, and many 
other names, is often treated as a simple extension of the accumulator; 
instructions are available for shifting back and forth within this extended 
register. The extended accumulator plays an important role in the 
multiply and divide instructions, as well as in the extended shift in- 
structions. The programmer with a particular machine at hand is 
referred to that machine’s manual for the details of the extended 
accumulator operations. 

The shift instructions have been discussed here in terms of arbitrary 
information processing. They are also important in fixed-point mathe- 
matical operations, since shifting is equivalent to multiplying by a 
power of the base of the computer’s number system. Thus shifting two 
places in a binary computer is equivalent to multiplying by 4. Again, the 
details of the use of shift instructions for arithmetic operations are 
intimately connected with the way in which the computer encodes posi- 
tive and negative numerical quantities. The details may be found in the 
instruction manuals for particular computers. 


7-4 Skipping 


An additional instruction is needed to examine a particular bit in the 
accumulator, in order to make a conditional decision based on a single 
bit. We might have some data that must be examined bit by bit, the 
value of each bit determining what is to be done next, An instruction 
that provides great flexibility is the following: 
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SKIP IF 1 IN BIT N. If the digit in bit position N in the accumu- 
lator contains a 1, then the computer skips the following instruction in 
sequence, going to the instruction 1 beyond. If the digit in bit position 
N in the accumulator is 0, the computer takes the next instruction in 
sequence. Bit positions are numbered 1 to 36 from left to right. 


There are several new features about this instruction. As in the case of 
the shift operations, the address field of the instruction is interpreted, 
not as the address of a register, but as a particular bit position in the 
accumulator. Moreover, the instruction is not a conditional jump in- 
struction, as all the other conditional instructions have been. Instead, 
if the condition is met, it causes the program to skip over the next 
register, omitting the instruction in that register. Of course the effect 
of a conditional jump instruction can be achieved by placing a jump 
instruction in the register following the skip instruction. Conditional 
skipping is a common practice in digital-computer hardware. On the 
other hand, an instruction that can examine any particular bit in the 
accumulator is very rare on digital machines. On most computers, 
programmers must “make do” with an instruction that will skip if the 
first bit is a 1 or one that will skip if the last bit is a 1. By shifting, such 
an instruction can be used to examine any bit. However, the more power- 
ful instruction that we have introduced on our hypothetical computer 
has the advantage of programming simplicity. 


Illustrative example: target detection. A particular hypothetical radar 
transmits a sequence of 35 energy pulses as it scans a certain sector of the sky. 
The radar echo from each pulse is examined, and if its energy exceeds a certain 
threshold, a hit is recorded; otherwise a miss is recorded. The sequence of 35 
pulses generates a 35-bit item, 1 bit for each pulse. The value of the bit is 1 
if the echo from the pulse was a hit; the value of the bit is a 0 if the pulse missed. 
If four successive pulses register hits, it is assumed that a target has been found. 
Otherwise it is assumed that the observed hits are due to random-noise var- 
iations. 

A subroutine is required that accepts a 35-bit entry in the accumulator as an 
input, the bits being in bit positions 2 to 36. The subroutine is to examine this 
record of radar returns to determine whether a target is present. The subroutine 
is to return to the main program with a 1 in bit position 1 of the accumulator 
if the input contained at least one target; otherwise it is to return with 0. The 
subroutine shown in Table 7-5 is to be entered by the instruction SET A AND 
JUMP TO DETECT. Since the subroutine needs one index register, it must 
save the contents of an index register so that it can be restored at the end of the 
subroutine. Index A is chosen for this purpose. In the main loop, bits will be 
shifted left, one by one, into position 1 in the accumulator, and that position 
will be tested for 1 or 0. As soon as there are no more Is in the accumulator, 
the routine is finished; so the test for completion is JUMP IF ZERO TO EXIT, 
Index A counts the number of successive hits; whenever a hit is detected, index 
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Table 7-5 Target-detection subroutine 


i ee EU UU EEE EEE EESSSSISIS SSIES HESIENEEIEIEEEIREn 





Location | Content of register Comments 
Program 
DETECT | STORE A IN SAVEA | Save content of A 
NEXT LOAD A FROM ZERO Start count of hits at 0 
JUMP IF ZERO TO EXIT Accumulator has hits 
and misses 
LOOP SHIFT LEFT 1 Look at next bit 
SKIP IF 1 IN BIT 1 Skip if hit 
JUMP TO MISS 
INDEX A FROM ONE Count successive hits 
JUMP TO LOOP 
MISS INDEX A FROM MFOUR | Is there a target 
(A > 4)? 
JUMP IF A MINUS TO NEXT If not, reset and continue 
COMPLEMENT Yes, target. Make bit 
1=1 
EXIT LOAD A FROM SAVEA | Restore original content 
of A 
JUMP TO 1:A Exit from subroutine 
SAVEA 0 Temporary storage 
ZERO 0 
ONE +1 
MFOUR —4 





register A is increased by 1 and the loop is repeated. Whenever a miss occurs, 
the value in index register A is examined. If it is less than 4, there was no target 
at this point; so the routine returns to reset A to 0 and continue looking at the 
remaining bits. On the other hand, if the value in index register A is 4 or more, 
then there is a target and it is necessary to return to the main program with the 
target indication. Since the determination of a target is made in the miss cycle 
of the loop, at the point of target detection the value in the first bit position 
in the accumulator is 0. Therefore, to make the bit 1 we merely complement 
the accumulator and exit. If there is no target, then the exit will occur from our 
test JUMP IF ZERO and in this case the accumulator will be all 0 and there- 
fore will fit the specification of having a 0 in bit 1. 


7-5 Input-output operations 


Input and output instructions vary widely from one computer to 
another, There are usually different instructions for each of the input and 
output devices associated with the computer, each with its own special 
idiosyncrasies. One common property of input-output instructions is the 
atomic detail they represent, To print out a single number composed of 
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several numerals may take a long sequence of computer instructions, 
including several output instructions. Mainly for this reason, program- 
ming input and output routines should be left to experienced pro- 
fessional programmers whenever possible. 

Inputs to computers are usually in the form of sets of alphanumeric 
characters, such as those typed from the symbolic coding forms. Like- 
wise, output from the computer is generally in terms of alphanumeric 
characters, either printed directly or recorded on tape for off-line print- 
ing. If punched paper tape is the input medium, each character is repre- 
sented by a set of punched holes in a row across the tape. On magnetic 
tape, the punched holes are replaced by recorded spots. Computer in- 
structions will generally be available to read a character from the tape 
and to write or punch a character on the tape. Our hypothetical machine 
includes the instruction READ CHARACTER to read one 6-bit alpha- 
numeric character from tape and place the code in the last six bit 
positions in the accumulator. The instruction WRITE CHARACTER 
records on tape the code in the last six digit positions of the accumulator, 
while the instruction PRINT CHARACTER prints the corresponding 
character directly on an on-line printing device. 

Since a computer word on our hypothetical machine may contain six 
alphanumeric characters, we shall often want to read in six characters 
and pack them in one word. This could be done by executing six suc- 
cessive read character instructions, interspersed with shift instructions. 
But because this process is needed so often, it is wired into our computer 
and is carried out automatically at the command READ WORD. 
Similarly the instruction WRITE WORD results in six successive write 
character instructions, with appropriate shifts interspersed. 

Computer instructions for punched-card operations are sometimes 
more complicated. Each character is represented by the punches in a 
single column on the card, but the machines read and punch the cards 
row by row rather than column by column. In reading, the patterns 
of bits in all the card rows must be stored and rearranged by a very 
complicated processing program to recover the column-coded characters. 
Punching requires the inverse process of computing the row-by-row bit 
patterns, or card image, corresponding to the characters which are to be 
the output. Sometimes these complicated operations are wired into the 
computer so that the card can be read as easily as the tape, but often a 
computer program is required. Special card-to-tape and tape-to-card 
machines are sometimes available for transmitting information between 
cards and magnetic tape. A row of the tape corresponds to a column of 
the card, and so the tape can be read and written in a straightforward 
character-by-character process, leaving all the rearranging to the special 
equipment, 
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Illustrative example: integer output subroutine. As a particularly 
simple example of an output subroutine, consider the problem of printing integers 
stored in the computer. A subroutine is required which accepts as input in the 
accumulator a positive binary integer and which prints out the decimal equiv- 
alent of this integer. (It is assumed for this example that all integers are smaller 
than 1 million, so that at most six characters are to be printed.) The subroutine, 
shown in Table 7-6, is to be entered by the instruction SET A AND JUMP TO 


Table 7-6 Integer output subroutine 


Location | Content of register Comments 
Program 
INTEG | STORE A IN SAVEA Save content of A 
LOAD A FROM FIVE Preset A for six 
digits 
STORE IN TEMP Save binary integer 
LOOP LOAD FROM TEMP Get number 


DIVIDE BY CONTENT OF TABLE:A | Get next digit 
STORE REMAINDER IN TEMP Save rest of numbers 
PRINT CHARACTER Print digit 





INDEX A FROM MONE Count down in A 
JUMP IF A PLUS TO LOOP Done when A — 
LOAD A FROM SAVEA Restore contents 
of A 
JUMP TO 1:A Exit from subroutine 
Constants 
SAVEA — Storage for A 
TEMP eH Temporary storage 
FIVE +5 Preset for A 
MONE —1 For indexing 
TABLE +1 Powers of 10 
+10 
+100 
+1000 
+10000 
+ 100000 
IN'TIEG. The subroutine needs an index register, and so it saves the value of 


index A, Index A is then preset for six digits. The input number is put into the 
accumulator and is divided by 100,000 during the first traversal of the loop. 
‘Then, for the purposes of this example, a special instruction is invented called 
STORE REMAINDER IN Y. This instruction stores the remainder of the 
division process as an integer in the specified storage register. The remainder 
occurs in the extended part of the accumulator in our hypothetical machine, 
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whereas the quotient is found in the main accumulator. The quotient will be 
an integer from 0 to 9, and we may assume for the purposes of this example 
that the alphanumeric codes for the numerals are their decimal equivalents; 
e.g., the binary code for the numeral 9 is the binary equivalent of 9, or 001001. 
Consequently we may execute the instruction PRINT CHARACTER after the 
division, in order to print out the value of the decimal digit. On successive 
traversals of the loop, the remainder from the preceding cycle is divided by the 
next lower power of 10 until on the final cycle division is by 10° or 1. 


Synchronizing the input and output mechanism with the central com- 
puter is partly a function of the computer program and constitutes an 
unfortunate complication in input-output programming. When an input 
mechanism is selected by the computer, characters are transferred to 
the computer at a rate determined by the speed of the mechanism. Each 
character in turn is transferred to a special input buffer register, and a 
signal is given to the computer that a character is ready to be read in 
from the buffer. The computer must execute a read character instruction 
to move the character into storage; then a signal is sent back that the 
buffer is ready for a new character. The computer must be able to accept 
the input information at least as fast as it is provided. If the computer 
fails to read a character before the next one is ready to be put into the 
buffer, the input mechanism will stop and time will be lost. In many 
cases, the mechanism is not able to stop instantaneously but must coast 
to the end of a fixed block of characters, so that some information would 
be glided over and lost. Card readers, for example, cannot stop in the 
middle of a card. The timing complication arises because the computer 
is generally very much faster than the input mechanism, so that merely 
executing a succession of read character instructions is wasteful of 
computation time. There is plenty of time between successive characters 
to do some initial processing. For example, a photoelectric tape reader 
might have an input rate of 100 characters per second, so that the com- 
puter must read a character once every 0.01 sec. However, at that rate 
a modern high-speed computer can perform more than 1,000 operations 
between characters. The programming problem is to have the computer 
spend most of the time between characters in useful computation but 
never to get behind the input mechanism. The particular way in which 
such synchrony is achieved depends on the details of each particular 
machine, but every machine has the problem in some form or other. 

Modern machines can be instructed to read and write blocks of com- 
puter words from successive storage registers. The computer program 
first sets up the information to be put out in a block of registers in what- 
ever output code is required. The program instructs the output mecha- 
nism to proceed to put out the block, by specifying the number of 
registers and the address of the first register in the block. The output 
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mechanism then proceeds at its own pace, and ‘the computer is free to 
continue with other calculations. This alleviates, but does not solve, the 
synchronization problem. One block of data can be processed while 
another block is coming in, but the program must be ready to switch 
blocks as soon as the next block is completely within the machine. 

Not all the input and output to the computer is in alphanumeric 
symbols. Arbitrary binary codes are sometimes used. A particular binary 
code that is in one-to-one correspondence with the internal storage of 
the computer is frequently encountered. The output of a symbolic 
assembly program, for example, is a tape or a deck of cards containing a 
one-to-one representation of the program as it will appear in storage. 
For binary machines, this means a set of the binary codes for every 
register in the program. When the program is to be run, this binary 
material is put into the computer by a special input routine called a 
loader. Input and output of binary material can of course be done with 
the read character and write character instructions, since these in- 
structions for a binary computer say, in effect, ““Read a set of 6 bits” 
or “Write a set of 6 bits.’”’ With binary information, however, the read 
word and write word instructions are more generally used. On some com- 
puters the loader must include special instructions to set the machine 
cither for binary information or for binary-coded characters. 

One input problem remains. As we have explained, data are read into 
the computer by subroutines in the program, and the program is read 
into the computer by a loader. But how does the loader get into the 
computer? The answer is that the loader pulls itself in by its bootstraps. 
‘he computer contains within itself a very small wired-in program to 
read in very simple programs. The wired-in program may do no more 
than put in two or three computer words from the input mechanism, 
after which it transfers control to this little two- or three-word program. 
‘This little program may be a loop that can read in the loader, which in 
turn can read in the program. There are two or three consecutive read-in 
processes, each a little more complicated than the other, starting from an 
exceedingly elementary read-in program, until the computer is in a 
position to do the complete read-in job. Reasonably enough, this 
procedure is known as bootstrapping. 

Check sums and parity bits. When information is read into the 
computer from an external source, like a card reader or a tape machine, 
there is always a chance that the equipment will make an error in 1| or 
2 bits. A dust spot on a magnetic tape or a card on which some holes are 
slightly out of line can cause the wrong value of a bit to be read. There 
is even a slight chance that a word will be transmitted incorrectly from 
one register to another within the machine. Checking procedures have 
been devised to detect such errors, 

Within the machine, transmissions can be checked by appending 
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an extra bit, called a parity bit, on each word. The value of the parity 
bit is set so that the total number of Is in the word, including this bit, 
is even. In our hypothetical computer, a 36-bit word containing twelve 
1s would have a parity bit of 0, while a 36-bit word containing thirteen 
1s would have a parity bit of 1. The parity bit is transmitted along with 
the other bits, and parity is checked automatically after each trans- 
mission. A parity alarm rings when parity fails to check. Most experi- 
mental, one-of-a-kind computers include automatic parity checking. 
However, some commercial computers make no internal parity checks 
because the circuits are felt to be so reliable that internal errors will not 
occur. Errors are much more frequent in the input and output processes. 

Parity checking is done automatically and does not have to be pro- 
grammed. The programmer’s only responsibility is to be prepared for 
the possibility of parity alarms. Sometimes the programmer wishes to 
include some checking procedures within his program, either because 
he does not trust the computer or because his computer does not check 
parity automatically. With modern computers, there is seldom any 
worry about internal processing, but there are always chances for 
machine errors in input and output. One common programming device 
that is often used in processing binary-coded punched cards is a check 
swum. When the card is punched by the computer—say, by the symbolic 
assembly program—the information on the card is grouped into com- 
puter words and each word is treated as a number. The numbers are 
added, and the sum, called a check sum, is put on the card as a separate 
computer word. If the sum overflows, as it usually will, the overflow is 
discarded. Then, when the information is later read into the computer, 
the check sum is calculated in the same way and compared with the 
check sum on the card. If the two check sums are the same, the infor- 
mation is taken as correct. 

Logically, a check sum or a parity digit is not an infallible indicator. 
Compensating errors could occur. If two digits were wrong in a computer 
word, the parity would nevertheless be correct. However, in practice, 
the chance of a single error is much greater than the chance of two 
compensating errors. If equipment trouble develops, so that errors be- 
come more probable, a spate of single errors is likely to precede any 
compensating errors and corrective action will have been taken. 


7-6 Odds and ends 


External control. It is frequently useful for the programmer to 
have control over the operation of his program by means of switches 
located on the computer console. For example, he might include some 
parts of his program that are useful only for debugging but that should 
be omitted when the program works properly, The program could be 
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arranged so that, if a particular switch on the console were set in a 
certain way, the entire program would be executed, whereas if the 
switch were set differently the debugging parts of the program would be 
omitted. The simplest kind of external control is an on-off switch. A 
computer might sense this switch by means of a skip instruction: SKIP 
IF SWITCH ON. Large computers will generally have several switches, 
each with its associated skip instruction. At the next level of complexity, 
the computer might be instructed to enter a complete word from the 
computer console. This word would be set up in a set of switches—36 
binary switches for the 36-bit registers for the hypothetical computer. 
Some computers have an on-line typewriter and can signal the operator 
that the computer is waiting for a typed input. The programmer can 
then type a prearranged set of characters that his program can sense. 

Stopping. An instruction is needed to indicate to the computer 
that the program is completely finished. This instruction is often called 
HALT. When it is executed, it lights an indicator on the computer’s 
control console, indicating that the computer has stopped. 


Problems 


1. A symbolic assembly program that translates programs written in symbolic 
form to machine-coded instructions needs several subroutines to manipulate 
alphanumeric characters. Suppose that an address may be symbolized by 
any set of one to six alphanumeric characters. Any symbolic address will 
then fit into a single register. 


a. A symbolic address with fewer than six nonblank characters may start 
in any character position in its register, so long as it fits into the register. 
Thus a four-character symbol can start in the first, second, or third 
character location. A routine is needed to place all symbols in a standard, 
left-justified form, starting at the left of the internal storage register. 
Write a subroutine that takes as input a word in the accumulator con- 
taining the alphanumeric codes for the six characters in a symbol and 
left-justifies this symbol by shifting the characters left one character 
at a time until the leftmost character is not a blank. Any character 
positions emptied by this shifting must be filled by the code for blank, 
which is 603. Thus, the computer changes **LOOP to LOOP**, where * 
stands for blank. The subroutine is to leave the left-justified symbol 
in the accumulator. 

b. The assembly program must assign an actual storage location to each 
symbol and must keep a table of these assignments. Each tabular entry 
consists of two registers, one containing the left-justified symbol, and 
the other containing the numerical value of the assigned address. Write a 
subroutine that takes as input in the accumulator the left-justified sym- 
bolic address and obtains from a register labeled LOC the address to 
be assigned to this symbol, The subroutine must add the symbol and 
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its assigned location to a table. Set up the table in any convenient form, 
using any convenient device to indicate how many symbols already exist 
on the table and therefore the location of the next available empty space 
for adding to the table. 


c. Write the companion subroutine to look up the assigned value of a symbol 


from this table. This subroutine takes as input the left-justified symbol 
in the accumulator and produces as output the associated numerical 
value in the accumulator. The routine exits to the second register follow- 
ing the register containing the transfer to the subroutine if a value is 
found but exits to the first register after the transfer if no tabular entry 
can be found. (Note that we may not automatically go to an error 
routine if no tabular entry is found, since we shall want to use this 
routine to test for possible duplicates before adding a new symbol to 
the table. That is, when a new symbol is encountered in the location field 
from the coding sheet, we shall first look it up in the table to be sure 
that it has not previously been defined, which would indicate an error, 
and then add it to the table. On the other hand, when we encounter 
this symbol in the address field of the instruction from the coding sheet 
then we shall expect to find it in the table, and not finding it will indicate 
an error.) 


Use symbolic notation in coding these subroutines. Assume that we al- 
ready have a symbolic assembly program but that we are writing a new 
and better version of it. 


. Twenty coded data words are stored in successive registers starting at 


DATA. Write a program that will examine these words to determine whether 
or not bits 3, 7, and 10 are 1s and bits 12 and 13 are Os. The locations of 
all words having the specified bit pattern are to be stored in successive 
registers beginning at FOUND, and the number of such words is to be 
stored in NFOUND. 

Expand the radar target-detection program in Table 7-5 to examine the 
records of hits or misses from 3,600 radar pulses stored in 100 successive 
registers. Assume that the pulses are numbered from 0 to 3599. Every set 
of four or more consecutive hits is to be considered a target. List the location 
of each target by specifying the number of the middle pulse in the set of 
consecutive hits. 


. Write a program that takes as input a computer word in the accumulator 


and produces as output a count of a number of Is in that binary word 
in the accumulator. 

A certain visual pattern can be reduced to a pattern of 36 bits and can thus 
be represented by a single computer word. Write a simple pattern-recog- 
nition program that compares each input pattern with a set of 10 stored 
patterns and determines the location of the stored pattern that the input 
most closely matches. Closeness is measured by the number of matching bits. 
Assume the existence of a subroutine like that of Prob. 4. In case of ties, 
resolve the problem arbitrarily. 


. The outcome of a certain psychophysical experiment is recorded continu- 


ously. For every 1-second interval, one of three events is recorded no response, 
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which is coded 0, correct response, which is coded 10, and incorrect response, 
which is coded 11. Each experimental trial generates a long sequence of 
binary digits, which are segmented into 36-bit words and stored in successive 
registers in the computer. In the analysis program, a subroutine is required 
that will produce the value of the next event. Write a subroutine that will 
produce 0 in the accumulator, indicating no response, 2, representing a 
correct response, or 3, representing an incorrect response. Indicate how the 
subroutine must be preset at the start of the analysis. Assume that the 
main routine knows how many time intervals have been coded so that the 
subroutine need not test for the end of the series. 


. Write a subroutine to pack six alphanumeric characters located in the last 


6 bits of six successive registers into a single computer word (the inverse 
of the program in Table 7-3). 


. Thirty-five integers are stored in 35 successive registers. They are to be 


reduced to a single 35-bit entry in a single computer word using bit positions 
2 through 36, each bit position representing a single number. A bit is set 
equal to 1 if the corresponding number is larger than a certain fixed threshold 
stored in the register labeled THRESH; otherwise the bit is set equal to 0. 
Write a routine to perform this transformation. 


. Using the instruction PRINT CHARACTER, which prints on the output 


device the character corresponding to the digital code in bits 31 through 36 
in the accumulator, write a subroutine to put out a block of alphanumeric 
information, starting with a certain register and proceeding for a number 
of registers. Each register contains six alphanumeric characters. The call- 
ing sequence for the subroutine must include information concerning the 
location of the information to be put out and the number of registers in- 
volved. Be sure to report the specification of the calling sequence for your 
program. 

An input routine has produced a computer word consisting of six alphanu- 
meric characters that specify a decimal integer. That is, the characters 
are codes for the numerals 0 through 9. Assume that the binary codes are 
the binary numbers corresponding to the decimal digits. Write a subroutine 
that takes such a word as input and produces as output the corresponding 
binary integer in the accumulator. 


. Write a more general conversion routine that accepts as input a word con- 


taining six characters, which may be either numerals or blanks (blanks have 
the octal code 60). A blank before any numeral is to be ignored; a blank 
following any numeral is to indicate that the integer is finished; a blank 
between integers is to be treated as an error. Assume a special error routine. 


. The decimal output routine in Table 7-6 has the slight disadvantage that 


it always prints six characters, even though the integer may be very small. 
For the integer 12 it will print 000012. Improve the routine by suppressing 
the initial 0s. Note that 0s in the midst of the integer must be printed and 
that if the integer is 0 then a single 0 must be printed. 
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Statistical calculations 
and data analysis 


| Beene and social scientists are often faced with the necessity 
of performing many tiresome calculations in order to determine 
the statistical relationships among the variables they are studying. 
Often the sheer volume of required calculations is so overwhelming as to 
preclude doing the analysis by hand. Before computers were available 
for doing the work, many researchers were forced to settle for half a 
loaf. Serious decisions had to be made about the potential value of an 
analysis as compared with its cost in manpower, time, and patience. 
With a computer to do the routine labor, a scientist can seriously con- 
template analyses that would otherwise have been too costly. A factor 
analysis that would have taken over a month to do by hand can be done 
in minutes on a computer. If a scientist is not sure which of two proce- 
dures to use, he can use both. If he suspects the existence of some obscure 
but interesting relationship, he can follow his hunches without fear 
of wasting months if the search proves to be fruitless. This is not to say 
that using a computer is effortless. But the work is much easier, shorter, 
and more pleasant. Sometimes a computer program can be found that 
will do what is wanted, but if not, a new program can be written about 
as easily as instructions can be listed for a human clerk, and, once writ- 
ten, the program can be used over and over again on new problems. 
‘The main function of statistical analysis is to help the scientist under- 
stand his data, Sometimes a scientist uses statistics only to prove a point, 
but then he seldom needs the help of a computer. An elaborate analysis 
is needed when there are many interrelated variables and when the inter- 
relationships are not clearly understood, There are many alternative 
ways to analyze a set of data, each showing a different facet of the rela- 
1 
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tions. With a computer at hand, a scientist can use more than one anal- 
ysis, and he can test alternative explanations of the data. Sometimes 
the result of an analysis will indicate that better results could be ob- 
tained by first making some transformation of the data. The computer 
allows the investigator to try several different transformations to see 
which is best. Of course this increased power must be used wisely. If an 
investigator looks hard enough and long enough at nothing, he is likely 
to think he sees something. That is, if many analyses are made indis- 
criminately, sooner or later one of them will turn up some circumstance 
in the data that may seem to be evidence of an underlying mechanism 
but is merely a chance occurrence. Used with discretion, however, the 
computer can help the scientist to understand his data thoroughly. 

If the computer is improving the lot of the practitioner by permitting 
him the freedom to perform more thorough analyses, it is having an even 
more profound influence on the theoreticians. Psychometricians and 
statisticians who design new methods of analysis have often been 
stalled by the impracticality of their inventions. They have had either 
to spend an unfortunate amount of their time devising practical proce- 
dures for calculating the necessary quantities or to run the risk of watch- 
ing their brainchild gather dust in a bin with other interesting but 
useless methods. Many times, the practical procedures have involved 
so many approximations that the extent of departure from the ideal has 
become in itself a formidable statistical problem, leading to a sense of 
futility not very different from that felt by those who avoided the prac- 
tical problems altogether. 

Now the developers of new techniques for analyzing data have the 
vast powers of computers at their disposal. Methods can confidently be 
proposed that require lengthy calculations, and procedures can even be 
suggested that have no formal solution at all but can be applied only by 
numerical approximations. Iterative methods, objective procedures, and 
Monte Carlo techniques, discussed below, can all be used to solve prob- 
lems that cannot conveniently be solved by direct methods. These 
methods are more than just tricks of numerical calculation—they are 
alternative ways of dealing with statistical problems and have freed the 
statistician from some of the constraints of formal mathematics. Shepard 
(1962), for example, has developed a multidimensional scaling method 
that seeks to find an appropriate data transformation by an iterative 
procedure. His method is much too cumbersome for hand calculations 
but is easily applied with a digital computer. Many more psychometric 
methods dependent on computers can be expected to be developed as the 
advantages of computers become more widely recognized. 

There is another side of the coin. If the methodologist now has the 
power of computers at his disposal, so does the scientist who wants to 
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use the methods. Before computers, there was a certain comfort in pro- 
ducing an elegant but impractical solution to a statistical problem. The 
elegant, formal solution was accepted as a scientific contribution, and 
no one, not even the author, knew whether or not his creation would 
really work. The computations were much too extensive and laborious 
for anyone to consider using the method on a real problem. Now a com- 
puter can be used to apply the method, and the methodologist had better 
be sure that his new method makes practical sense, lest others use his 
method only to discover that it does not. The methodologist now has 
the responsibility of ensuring that his method works before he publishes 
it. 

This chapter will not discuss the many techniques that can be applied 
efficiently with digital computers. There are too many; besides, such a 
discussion belongs in a book about statistics. Instead, the chapter will 
consider some of the special issues that arise in performing numerical 
calculations on digital computers. First some comments are made about 
making standard statistical analyses. The general problem of the accu- 
racy of digital calculations is discussed, and the use of approximations 
is described. Then, some special procedures are given that digital com- 
puters have made practical: iterative procedures, hill-climbing proce- 
dures, objective procedures, and Monte Carlo procedures. Finally, some 
special uses of computers in data analysis are touched upon: on-line 
continuous data analysis and clinical diagnosis. 


8-1 Standard statistical analyses 


Any analysis that can be handed to a statistical clerk can also be handed 
to a computer, with a little additional effort. A programming language 
like ForTRAN permits the details of the analysis to be expressed about 
as easily for the computer as for the clerk, although it is sometimes more 
difficult to tell whether or not the computer is doing the right thing. 
New statistical programs have to be checked by using a small set of data 
and comparing the machine’s output with the results of a hand calcula- 
tion. But if the data for checking are carefully contrived, most of the 
program’s features can be tested with little effort. 

Data analysis is facilitated, for many machines, by the existence of 
extensive libraries of statistical routines. (See Dickman’s description of 
the statistical library for the Inu1ac at the University of Illinois, 1959.) 
Programs are commonly available, not only for simple ¢ tests and chi- 
square tests, but also for least-squares curve fitting, multiple regression, 
analysis of variance, factor analysis, probit analysis, and many psycho- 
logical scaling methods. linding an existing program can be difficult. 
The best way, as always, is by personal contact with the program’s 
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author. Statistical programs are regularly reported in The Digital Com- 
puter Newsletter, a department of the journal Behavioral Science, and also 
in the journals Educational and Psychological Measurement, Techno- 
metrics, and Con 7 unications of the Association for Cow puting Machinery. 
Statistical programs are often distributed through the users’ organiza- 
tions of each computer model, but many scientists for one reason or 
another do not distribute programs through these groups. 

When a program cannot be found and must be written, an algebraic 
compiler like Fortran greatly facilitates the work. The illustrative ex- 
ample in Chap. 5 shows that writing the formulas for the means, var- 
iances, and intercorrelations for a set of variables is done very easily in 
an algebraic language like Fortran. Many of the statistical techniques 
widely used in psychology are based on simple manipulations of matrices 
of numbers. A matrix is just a rectangular array of numbers, but it is 
especially convenient for arranging calculations. Multiple regression, 
factor analysis, psychological scaling, and analysis of variance can all 
be written in terms of matrix notation (see Gulliksen, 1959; Torgerson, 
1958). The calculations for these procedures involve matrix addition, 
matrix multiplication, matrix inversion, and sometimes the determina- 
tion of the latent roots and vectors of a matrix. Matrix inversion is the 
counterpart of ordinary division and is fundamental to the solution of 
simultaneous linear equations, which appear in many applications. Deter- 
mining the roots and vectors of a matrix is a companion procedure that 
occurs commonly in problems involving the minimization of the sums 
of squared quantities. In the language of an algebraic compiler, matrix 
addition and multiplication may be written almost immediately, as 
shown in Table 8-1. Programs for inverses and for roots and vectors are 


Table 8-1 FortTRAN routines for matrix addition and multiplication} 


Matrix Addition (A + B= C) 


DO 10 I=1,NI 
DO 10 J=1,NJ 
10 C(I, J) = Ad, J) + Bd, J) 


Matrix Multiplication (C-D = E) 


DO 20 I=1,NI 
DO 20 K=1,NK 
E(I, K) = 0.0 
DO 20 J=1,NJ 
20 E(I,K) = E(L, K) + C(I, J)*DG, K) 


+ In these computations, subscript I ranges from 1 to NI; J from 1 to NJ; K from 
1 to NK. Matrices A, B, and C have NI rows and NJ columns; D has NJ rows and 
NK columns; FE has NI rows and NK columns, 
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more complicated, but they are used so frequently and universally in all 
branches of science and engineering that every computer is sure to have 
such routines available in its library.! 

For the vast majority of numerical computations, the computer may 
be used in a very straightforward manner. For detailed discussions of 
numerical analysis the reader is referred to any of the number of fine 
books on the subject, including Hildebrand (1956), Householder (1953), 
Dwyer (1951), and Milne (1949). 

Graphs. A digital computer not only can calculate the required 
tables of numbers for analysis but can also draw graphs of appropriate 
quantities, if the required output devices are present on the computer. 
As described in Chap. 10, many computers have a cathode-ray tube out- 
put that permits the computer to draw a TV-like picture and therefore 
to make a graph of any required quantities. An automatic camera 
attached to such a display can record this graph on photographic film. 
Other computers have either an on-line or an off-line plotting device that 
will plot graphs in ink directly on paper. Of course, if only a few graphs 
are wanted, it is much easier to do them by hand than to write the nec- 
essary computer programs, but if a large number of graphs must be 
made, then an automatic device can be very handy. 

Data preparation. Statisticians are fond of warning experimental- 
ists to plan the statistical analysis before doing the experiment, so that 
the data can be collected in a manner that will simplify the analysis. 
‘The same warning applies with double force to analyses that are to be 
done on computers. The data should be convenient for both the statistics 
and the computer. If the data are tabulated on paper, there should be 
ample identifying material so that the numbers will not get mixed up 
when the data are transcribed to punched cards or punched tape. With- 
out some forethought about the problem of transcription, the experi- 
menter may be faced with the task of retabulating his data manually 
so that it can be conveniently transcribed. 

Automatic devices for recording data are good investments for large 
quantities of data. Machines can be set up to record data directly onto 
punched cards or punched tape. The machines can be actuated by the 
subject’s response, such as the pressing of one of a set of response buttons 
or levers. More elaborate transducers can be set up to record other 
activity. For example, a rat’s course through a maze can be recorded 
automatically by means of microswitches hidden beneath the floor- 


' Many matrix problems in psychology reduce to the simplest latent root equation, 
(A = \I)x = 0, where A is a real, symmetric matrix. Some problems, including 
multiple discriminant functions and multivariate analysis of variance, require a more 
general equation, (A = \B)x « 0, where A and B are symmetric matrices. Most 
standard routines solve the former problem as a special case of the latter. 
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boards. Time can be recorded simultaneously with the other response 
indicants. Many elaborate data recorders that now produce a pen re- 
cording could readily be augmented to produce a punched record. If a 
fully automatic data recorder seems too elaborate or too expensive, some 
efficiency may be gained by a semiautomatic system in which the experi- 
menter uses a keyboard to record the data directly in punched form, 
rather than using a pencil to make a written record that must later 
be transcribed. 


8-2 Accuracy 


It is always important in numerical calculations to consider how many 
significant figures are required in the final answer and to use enough 
extra digits in the computations so that the final result will have the 
required degree of accuracy. In hand calculations, a human computer 
can use his common sense in keeping an appropriate number of digits 
and can adjust this number in the midst of the computations if something 
unexpected occurs. A computer program has no common sense; all rules 
for keeping significant digits must be provided as part of the program, 
and all contingencies must be anticipated. 

In the vast majority of cases, the required accuracy can be achieved 
automatically by using floating-point operations. As described in Chap. 
6, a computer word encoding a floating-point number has two parts, a 
fraction and a scale factor. The fraction always contains the most sig- 
nificant digits in the number, and the scale factor indicates the location 
of the reference point—the decimal point in the decimal system, the 
binary point in the binary system. The number of digits in the fraction 
is enough to give sufficient accuracy for most problems. On our hypo- 
thetical machine, the fraction of a floating-point number contains 27 
binary digits, which are equivalent to about 8 decimal digits. (21° = 
1,024, or approximately 10*; so 10 binary digits are about equal to 3 
decimal digits, whence the exchange rate of 3.3 binary digits per decimal 
digit.) 

Using floating-point numbers will usually provide enough accuracy, 
but some care must be taken in arranging the calculations so that the 
required accuracy of an intermediate result never exceeds the capacity 
of the floating-point representation. Thus, if two quantities are nearly 
equal, their difference will have fewer significant digits than either. 
For example, if the variance of n numbers is obtained by first subtracting 
the square of the grand total from n times the sum of squares of the 
numbers, and then by dividing this difference by n’, i.e., 
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the accuracy of the numerator will suffer slightly if n is large. It is 
better to obtain the mean and the mean square and then to calculate the 
variance by subtracting the squared mean from the mean square, i.e., 


“13-3 


Note that the former method is better if n is small, since only one division 
is involved, but this advantage exists only when all the digits in the 
numerator can be kept. 

The best way to maintain accuracy is to make sure that all the digits 
are doing their part. If all the scores on a test are between 100 and 150, 
the hundreds digits is excess baggage that will inflate sums of scores and 
sums of squared scores to no purpose. Arbitrarily subtracting 100 from 
each score will make the range of scores more tractable. Likewise, in a 
large analysis of variance the sums of squares can be kept within bounds 
by first subtracting from each data value the grand mean or some con- 
stant nearly equal to the grand mean. 

Some programmers are hesitant to use floating-point calculations be- 
cause floating-point operations are reputed to be slower than their 
fixed-point counterparts. This reputation is only partly deserved. Float- 
ing-point addition and subtraction are slower, but the multiplication and 
division speeds are about the same in the two systems. Of course some 
computers do not have built-in floating-point operations and must use 
subroutines to perform floating-point calculations. Fixed-point calcula- 
tions go much faster on these machines, but even so the advantage in 
speed is lost in the extra debugging needed by a fixed-point program. If a 
calculation is done with fixed-point operations, the programmer must 
carefully consider all the accuracy problems, as well as all the scaling 
problems involved in keeping track of the reference point. These head- 
aches should be left to professional programmers. 

If more accuracy is needed than normal floating-point numbers can 
provide, double-precision procedures must be used. Two registers are 
used to store each number in the computer, the number of significant 
digits that can be kept being thus doubled. In cases of dire need, triple 
precision can be used, with three registers per number. Multiple-precision 
calculations are much slower than single-precision calculations, because 
complicated subroutines must be used for all operations, including addi- 
tion, subtraction, multiplication, and division. (The very modern IBM 
7044/7094 computers have a built-in facility for double-precision arith- 
metic.) Even on computers requiring subroutines for floating-point oper- 
ations, multiple precision is slower, because the subroutines are longer. 
The use of double precision or triple precision in numerical calculations 
also introduces a large number of untidy details, There will be many 





























148 Behavioral-science applications 


fewer utility programs for dealing with the more precise quantities. 
Most algebraic compilers are not able to handle double-precision num- 
bers; so the entire program must be written in machine language. 
Double precision can be used with either floating- or fixed-point num- 
bers; the routines are untidy in either case. 

Sometimes there is no way to avoid the use of extra precision. For 
example, matrix inversion will require multiple precision if some of the 
latent roots of the matrix are suspected of being vanishingly small, so 
that the determinant is nearly zero. Of course, the result will be unstable 
in this case, quite apart from its precision, but the precision is still 
needed. In most applications, however, multiple precision can be avoided 
by using floating-point numbers and by properly arranging the calcu- 
lations. 


8-3 Approximations of standard functions 


The so-called “transcendental” functions occur frequently in mathe- 
matical problems in psychology. Sines, cosines, and other trigonometric 
functions appear in problems of vision and visual perception. Logarithms 
and exponents are commonly used in analyzing data from experiments 
in audition, and in psychological scaling. Normal deviates are employed 
in many parts of psychometrics. A human calculator would look up the 
values of these functions in a table, but a computer can calculate a 
value for one of the functions more conveniently than it can store a 
large table. 

The computer calculates values of a transcendental function in the 
same way that the tables of the function were made originally: by a 
series approximation. The series are generally polynomials of the form 


f(e) = ao + ae + agx? + agar? + aie 


The approximations are set up so that successive terms in the series 
make smaller and smaller corrections to the value; the number of terms 
one uses depends on the required accuracy. In the above series, successive 
terms will decrease if x is less than 1. Usually four or five terms in the 
series will provide a result correct to five or six decimal places, which is 
more accurate than most standard tables. 

Nearly all computers will have library subroutines for calculating the 
transcendental functions. These routines are generally available to a 
program through the symbolic assembly program, so that the program- 
mer can put a single instruction in his symbolic program that will cause 
the subroutine to be assembled with his program. Algebraic compilers 
have similar arrangements. In Forrran, merely writing SINF(X) as 
part of an arithmetic expression causes the library subroutine for sines 
and cosines to be included in the compiled program. 
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If a series approximation to a function must be developed from 
scratch, the values of the coefficients can be determined by a straight- 
forward Taylor series expansion, which is described in any calculus text. 
The coefficients must sometimes be modified slightly so that the approxi- 
mation will work reasonably well over the entire range of values con- 
templated. The subtleties of such modifications and a great many handy 
series are contained in Hastings’ excellent book ‘Approximations for 

Digital Computers” (1955). 


8-4 Iterative procedures 


An approximation, if it is close enough, is as good as an exact solution, 
and some numerical problems cannot be solved exactly. Other problems 
have an exact solution that is long and intricate. Many of these prob- 
lems can be solved approximately by straightforward, tedious, iterative 
procedures. The required accuracy of the approximation can be attained 
by performing enough iterations. Since computers are so swift at doing 
straightforward, tedious tasks, iterative solutions are easily achieved 
on computers and are sometimes preferable to a complicated exact 
solution. 

An iterative process generally involves making an initial estimate of 
the solution and then using some equation to provide a better estimate. 
The better estimate is substituted in the equation to provide a still better 
estimate, and so on, until successive estimates differ from each other by 
a very small amount, within the required limits of accuracy. As a simple 
example, consider the problem of computing the square root of a number. 
Most readers will dimly recall the complicated algorithm they were 
taught in secondary school for determining square roots. Other intricate 
procedures are available for calculating square roots on a desk calculator. 
But the digital computer prefers a very simple iterative procedure that 
dates back to Newton’s time. The iteration is expressed in the equation 


1 x 
Yiu = 2 (v + *) 


where x is the number whose square root is sought and the ys are suc- 
cessive approximations to that square root. Initially a guess at a square 
root is made. For example, we could guess that the square root of 2 is 1. 
Substituting this guess for yo and 2 for x in the equation, we obtain the 


next estimate, 
1 2 
w= (1 + i) = 1.5 


Now, if we specify five-place accuracy, we require that the absolute 
value of yin = yi be less than 0,00001, Since y= yo = 1.5 — 1 = 0.5, 
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another iteration is required. Table 8-2 shows the successive estimates of 


V2. This procedure will find a square root as accurately as any exact 
algorithm, except in the case of exact squares. 


Table 8-2 Successive estimates of V2 


Yo 1.000000 
1 1.500000 
Yo 1.416667 
Ys 1.414216 
Ys 1.414214 


Iterative methods are especially useful in multivariate problems. In 
factor analysis, for example, the best solution to the so-called ‘“‘ccommu- 
nality” problem is by iteration. The problem arises because the factor 
loadings cannot be estimated without knowing the communalities (ex- 
cept in Lawley’s method, 1940), but the communalities are also un- 
known, being the sums of squares of factor loadings for each test. The 
iterative solution involves making initial estimates of the communalities 
by some convenient rule and using these estimates to extract the factor 
loadings, perhaps adjusting the communalities along the way. Whena 
decision is made about the number of factors to be retained, the factor 
loadings for these factors provide new estimates of the communalities 
and new factor loadings can be obtained. The process continues until 
none of the communality estimates change by more than a preassigned 
amount. 

There is no a priori guarantee that an iterative method will converge. 
Possibly the successive solutions will never stop changing, and the 
changes, instead of getting smaller and smaller, may get larger and 
larger. The convergence of an iterative procedure can sometimes be 
determined by a simple mathematical analysis, the details of which can 
be found in Hildebrand (1956) and elsewhere. Usually an iterative proc- 
ess is a fancy kind of averaging process and is bound to converge, but 
sometimes trouble arises. The square-root iteration explained above will 
always converge but is, of course, valid only for positive x. If a negative 
number is put for x, the process diverges in an oscillating manner. So far 
as the author knows, there is no definite proof that the communality 
iteration converges, but except for some bizarre cases it seems very 
likely that it will converge. 


Illustrative example: the Jacobi method.' An interesting example of 
an iterative technique is the Jacobi method for calculating all the latent roots 


and vectors of a matrix. Here we shall discuss the common case of a square 


1A knowledge of matrix algebra is required for this illustration, 
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symmetric matrix A. The method obtains all the roots at the same time and is 
indifferent to multiple and zero roots. 

The idea is remarkably simple. On each step of the iteration the matrix is 
transformed by a two-dimensional rotation so that one of its off-diagonal ele- 
ments becomes 0. Successive transformations are made until no off-diagonal 
remains. Specifically, on the first iteration we choose an off-diagonal element 
of A, say, aij, that is to be reduced to 0. To accomplish this, we premultiply and 
postmultiply A by an orthogonal transformation matrix Bo of the same size as A. 


Ai = BAB,’ 


where Bog is the transpose of Bo. Bo is the identity matrix except for the four 
elements by, 6:;, bj:, and b;;. They are determined so that ai; = 0. Grinding 
through the algebraic equations we find that 








1+s 
bss = bi = Af 
1—s 

bi = —Op = ; 
where s= a ae 
V1i+? 
and t= a 
Ajj — Gi 
(In the special case when aj; = aii, s = 0.) 


For the next iteration, another off-diagonal value of Aj is chosen, a corre- 
sponding B; is calculated, and A» is computed, 


As = B,A,B{ = B,B,ABjB{ 


The process continues until all off-diagonals are less than a specified small 
amount. Note that there is nothing to prevent an off-diagonal term that was 
made zero on a previous iteration from beginning nonzero on a future iteration, 
but eventually all the off-diagonals will be beaten down. The final result, say, 
A,, is a diagonal matrix, containing the complete set of roots. Multiple roots 
or zero roots do not embarrass the Jacobi method as they do some other methods. 
If some roots are zero or are equal to other roots, they will appear as such in the 
matrix Ay. The product 


n 
A= II B; 
i=0 


ix the matrix of latent vectors such that 


A = AA,A’ 
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Ideally, each iteration should attack the largest remaining off-diagonal, but in 
practice it is often swifter to scan the matrix, dealing with each off-diagonal 
in turn, and then to scan all the off-diagonal elements a second time, or a third 
time, if necessary. 


8-5 Hill-climbing procedures 


A special iterative technique known as hill climbing is often useful in 
applying mathematical models to data. A mathematical model is an 
acceptable account of the data at hand if it can be adjusted so that the 
predictions of the model are in close agreement with the actual data. 
The adjustment, or “‘fitting,” usually takes the form of choosing values 
for certain parameters in the model so that the predictions of the model 
match the data. Sometimes the proper choice for the value of a parameter 
can be determined directly from the data. For example, the value of 
some parameter might be the average of all the data values. If no direct 
method can be devised for computing the best parameter value from the 
data, an iterative solution may be possible. As described in the preceding 
section, initial estimates of the parameter values are made and iteration 
equations are used to determine better estimates. The resulting estimates 
can be put back into the iteration equations to derive even better esti- 
mates, and so on. But for some models even the iteration technique fails. 
There may be no way to set up iteration equations. In these unfortunate 
but not infrequent cases, the parameter values must be chosen by trial 
and error. Different sets of values of the parameters are tried, and the 
fit of the model is determined for each set. The best set is taken, and 
further changes are made in the values in an attempt to improve the fit, 
until no further improvement can be made. When trial-and-error esti- 
mation is done in a systematic way, it is called hill climbing. 

The name hill climbing comes from a way of visualizing the problem 
in the case of two parameters. We can depict the range of values for the 
parameters by a graph, with one parameter as the abscissa, or horizontal 
coordinate, and the other as the ordinate, or vertical coordinate. The 
graph is like a map, with one parameter running north and south and 
the other running east and west. Any choice of values for the parameters 
corresponds to a particular point on the graph, and conversely any point 
on the graph corresponds to a particular pair of parameter values. Now, 
each such pair of values completes the model, so that predictions can be 
made and compared with the actual data. We assume that the differ- 
ences between the predictions and the data can be measured, so that the 
fit can be expressed as a single number. The larger the number, the better 


the fit. The fit can be obtained for any pair of parameter values, and — 


hence for any point on the graph. To depict the fit, we need a third 
dimension—say, the height away from the graph paper, or the height 
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of the terrain at that spot on the map. If we were to determine the ‘fit 
of the model for every possible point, we would obtain a surface of vary- 
ing height all over the graph. Our problem is to find the highest point 
on this surface, or the highest hilltop on the map. If there are more than 
two parameters, the surface is multidimensional, each parameter being 
a dimension, and geometric visualization becomes difficult. But we can 
still talk about the problem in terms of climbing a hill to its top. 

Hill climbing differs from other iterative schemes in the way by which 
values of the parameter are improved. There are no equations into which 
the current values may be substituted to yield better values. All that 
can be determined is the height of the surface at the point corresponding 
to the current parameter values. Some extra rule is needed for deciding 
how to change the values. A procedure is needed for probing the surface 
to locate peaks, since it is usually not feasible for the computer to cal- 
culate the value of the function at all points. 

Several methods can all be subsumed under the general title of hill 
climbing. Starting at a point somewhere near the maximum, one or more 
neighboring points are examined, and depending on the height of the 
function at those points, a new value is chosen and the process continues. 
The simplest procedure would be to examine any adjacent point obtained 
by changing the estimated value of only one parameter. If an improve- 
ment is noted, we step to the new point and continue “walking” in that 
direction until no further improvement is possible. Then another param- 
eter is changed; i.e., another direction of walking is tried. If improve- 
ment results, that direction is kept until no further improvement occurs; 
if improvement does not result, the reverse direction is tried or some 
other parameter is changed. This gradual step-by-step movement will 
eventually climb the hill to its peak. 

The method of steepest ascent (often called descent, in the framework 
of valleys rather than hills) is a cautious hill climber. Every possible 
direction is considered, rather than just locations corresponding to a 
change in a single parameter. The direction yielding the most improve- 
ment is chosen. To use the method, one must be able to calculate the 
direction in which greatest improvement is expected, since all possible 
directions cannot be examined empirically. 

Any hill climber will find a peak. If the function is smooth and has only 
one peak, then the correct solution will be obtained. But if the function 
has several peaks, the hill climber might climb the wrong hill, finding 
only a local maximum. Some insurance must be provided against the 
location of local peaks. Several possibilities exist. Large steps can be 
taken at first, and the step size can be reduced gradually as the peak is 
approached, Or, using the method of direct search (Hooke and Jeeves, 
1V61), one can pause every now and then and probe some other part 
of the domain, chosen in an arbitrary way, just to see if there are other 
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hills. One can precede the hill-climbing phase by a search phase in which 
the height of the function is first determined for each of a grid of points 
spanning the space of permissible parameters. Perhaps two or three 
promising points can then be tried as bases for hill-climbing expeditions. 

A more serious problem, from a statistical point of view, is in deter- 
mining the steepness of the hill. It has often been observed in practice 
that the peak is more of a plateau or mesa, the maximum height not 
being very different from a great many neighboring points. Any hill 
climber will find the maximum, but something like the grid procedure 
is needed for determining the range of solutions that for all practical 
purposes can be considered maximum. The existence of a plateau is 
also important in understanding the model. The plateau implies that 
many of the parameters in the model do not have a very significant 
effect or that there are many compensating factors in the model, a 
change in one parameter being easily offset by a compensating change 
in another. A better model is called for, not a better estimation procedure. 

Hill-climbing procedures require computers. Calculating the fit of the 
model with a specified set of parameter values is likely to be a big chore, 
and the notion of doing this over and over again with slightly different 
values of the parameters is realistic only when computers can do the 
job. Before computers, the estimation of parameter values by trial and 
error seemed like a desperate measure, and although it is still only a last 
resort, at least it is not an unreasonable procedure with a computer. 

Hill climbing is also a useful procedure in a slightly different context. 
Computer programs that play games, solve problems, and exhibit other 
“intelligent” behavior, as discussed in Chap. 11, often include some 
mechanism for automatically improving their performance. Some pro- 
vision is made for evaluating the program’s behavior, and the evaluation 
is used in the same manner as the quantity measuring the fit of a model 
in a regular hill climber. That is, the evaluation gives the height of the 
surface, and the computer program makes changes akin to altering 
parameter values, in an attempt to get a higher evaluation. For a dis- 
cussion of hill climbing in this context, see Minsky (1961a). 


8-6 Objective procedures 


Some problems can best be solved by procedures that are objective but 
that cannot easily be expressed as a set of mathematical equations. The 
best test of the objectivity of a process is a computer program. If the 
process can be programmed for a computer, the analysis can proceed as 
rigorously as in the case of the direct solution of the equations. Hill 
climbing is a case in point. Maximizing a multidimensional function by 
a hill-climbing process cannot be described solely by a set of equations, 
Some procedural statements and contingent rules are required, In short, 
the process is most naturally stated as a computer program, But a com- 
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puter program is a completely objective process. Once the rules have 
been specified, the hill climber will climb the hill the same way every 
time. Any shortcoming of the hill-climbing procedures is not the fault 
of objectivity. To be precise, it must be admitted that mathematical 
logic permits us to express any objective process in a set of equations 
and mathematical statements. In fact, any computer program can be so 
expressed. But such expressions are seldom amenable to further mathe- 
matical analysis; so the computer program remains the most practical 
objective statement of the process. 

The distinction between an objective process and a set of equations 

can readily be made in the problem of axis rotation in factor analysis. 
The parameters of the factor model cannot be determined uniquely 
without some additional specifications. Given one set of parameter val- 
ues, a multitude of other sets, all equally good, can be obtained by a 
process called rotation of aves. Some additional criteria must be estab- 
lished for choosing the “best”? among this multitude. In the past, the 
criteria have been subjective. A desirable set of parameters should have 
zero values for many of the parameters and positive values for most 
of the others, but specifying how many should be zero and how many 
positive was left to the discretion of the user. In short, practitioners did 
not take the trouble to specify exact rules for rotation of axes. The 
analytic rotation procedures currently in vogue solve the problem of 
subjectivity by writing a single equation embodying the rotation rules. 
While objectivity has been achieved, to be sure, the analytic nature of 
quartimax, biquartimin, varimax, and the rest, is of little practical value, 
because the equations are too complex to be analyzed mathematically. 
The properties of these rotation methods must still be determined 
empirically. 

The middle ground is a computer program that selects variables to 
define axes according to fixed, specified criteria. Tucker’s (1944) semi- 
analytic method of rotation is nearly objective and could easily be made 
completely so. He chooses subsets of variables to define factors according 
to criteria that are either completely objective or that can be stated as 
objective rules for a computer. Cattell and Muerle (1960) have reported 
a computer program that simulates the behavior of a factor analyst 
choosing new rotations for an oblique simple structure. So long as the 
rules for making rotations are sufficiently precise that a computer pro- 
gram can follow them, the resulting solution is completely objective 
and is as useful as one of the analytic solutions. 


8-7 Monte Carlo procedures 


When a statistical procedure is so complicated that its properties cannot 
readily be discovered by analytic methods, empirical methods can be 
used, Random numbers are used to construct hypothetical data that 
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conform with the statistical model. The statistical procedure in question 
is applied to many sets of such data, to provide estimates of the reliabil- 
ity of the results. With a digital computer to do the routine labor, the 
procedure can be tested in many different situations to give a compre- 
hensive picture of its properties. 

Computations based on random numbers are called Monte Carlo com- 
putations, as a tribute to the patriarch of random processes, the roulette 
wheel. Computers do not have roulette wheels, but they do have other 
sources of large quantities of random numbers (see Chap. 9), and they 
also have the untiring capacity to do the millions of necessary calcula- 
tions. Random numbers are sometimes used in iterative numerical 
schemes for solving large systems of equations, and these too are called 
Mcnte Carlo computations. But in the behavioral sciences Monte Carlo 
usually signifies an empirical study of a method or model. 

Stochastic models of behavior often depend on Monte Carlo methods 
fcr studying their consequences. Some of the Bush-Mosteller (1955) 
learning models have parameters for which neither expected values nor 
probability distributions can be calculated directly. Instead, the model 
must be simulated by using random numbers. A hypothetical learning 
experiment is run, using hypothetical organisms (which Bush and 
Mosteller call stat-rats) following the acquisition equations of the model. 
The resulting data are employed to calculate such quantities as the 
average number of trials to some criterion of learning performance. By 
running many such experiments with the same model, a distribution 
of this average value can be built up. 

Even with a digital computer, a Monte Carlo determination can be 
very time-consuming. Running the analysis or the model on one set of 
data can be a big task, since a large number of data are needed to provide 
an accurate result. On the other hand, any result is subject to error, 
and enough independent runs must be made to secure a good estimate 
of the variance involved. Anyone planning a Monte Carlo determination 
must carefully apportion his effort between making each run large 
enough for accuracy and making enough independent runs to provide 
information about variability. 


Illustrative example: statistical learning model. In one form of the 
statistical model for learning proposed by Bush and Mosteller (1955) the prob- 
ability of a correct response, p, is altered in one of two ways in every trial, 
depending on the outcome of that trial. If the correct response occurs, p is aug- 
mented by the quantity ai(1 — p) — bp, while if the incorrect response occurs, 
p is changed by the quantity ax(1 — p) — bop, where generally az and by are 
chosen such that the change is a decrement. The behavior of this model is to 
be simulated on a computer. For each set of parameters a), 6), dy, by, and po 
(the initial value of p) supplied by the programmer, the computer is to run 10 
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sequences of 100 trials each, simulating the behavior of 10 organisms. The flow 
diagram for the program is shown in Fig. 8-1. 


Illustrative example: the effects of violations of assumptions under- 
lying the ¢ test. The statistical significance of the difference between the 


means of two distributions can be determined by a ¢ test, if the two distributions 


START 


Read in parameters a,, by, ao, bo, Do 


Preset for 10 sequences 


Preset for 100 trials 


Choose a random number between 0 and 1 


ls random number less than p? 
Yes No 





Probability of this 
event is p 


Probability of this 
event is 1—p 


Calculate new value of p: 
new p=p+a,(1-p)—bp 


Calculate new value of p: 
new p=p+a,(1-p)—bep 


Print out and/or 
graph new value of p 








Nd Add one to count of trials. Have 100 trials been done? ) 


Yes 


No Add one to count of sequences. Have 10 sequences been done? 


Yes 





STOP 


Vig. 8-1, Flow dingram for Monte Carlo run of stochastic learning model. 
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are samples from normal distributions with the same variance. The consequences 
of applying the ¢ test when the underlying distributions are not normal or have 
different variances cannot easily be determined theoretically. Boneau (1960) 
has reported an empirical evaluation using a Monte Carlo technique on an 
IBM 650 computer. He applied the ¢ test to artificial data constructed according 
to a variety of specifications, including three different types of distributions, 
two different standard deviations, and two different sample sizes. 

Here we shall consider only the condition in which both distributions were 
normal, with zero mean, one distribution having a standard deviation of 1, 
the other a standard deviation of 2, and we shall consider samples of five from 
each distribution. The computer program is so straightforward that no flow 
diagram will be shown. The computer first obtained a sample of five values 
drawn at random for a normal distribution with zero mean and unit standard 


Frequency 








Value of t 


Fig. 8-2. Empirical values of ¢ obtained from Monte Carlo tests of the difference 
in the mean of two normal distributions with different standard deviations. (By per- 
mission from C. A. Boneau, “The Effects of Violations of Assumptions Underlying 
the t-test.”’ Psychological Bulletin, 1960.) 


deviation, directly from a subroutine that generated random normal deviates 
(see Sec. 9-3). Then a second such sample was drawn, and each value multi- 
plied by 2, to provide a sample from a normal distribution with a standard 
deviation of 2. The ¢ test for the difference between the means of these two sam- 
ples was computed, and the value of ¢ was tallied in a distribution. The entire 
process was repeated 1,000 times, yielding the distribution shown in Fig. 8-2. 
A total of 6.4 per cent of the obtained ts exceeded the theoretical 5 per cent limits. 


8-8 On-line continuous data analysis 


A digital computer is sometimes rapid enough to analyze data as they are 
being collected. Two properties of computers make such uses possible. 
First, they are very much faster than the data generators; second, they 
can synchronize their behavior with the input equipment, performing 
many operations between successive inputs, Discrete data can be read 
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as digits into the input buffer register of the computer and thence into 
the computer storage. Analog data can be put through an analog-to- 
digital converter before going to the input buffer. If the data come at a 
slow enough rate, the computer has time between successive inputs to 
do some simple calculations. A subject in a psychophysical experiment 
could easily be provided with a running account of his hits and misses 
by even a slow drum computer. A fast computer could calculate the 
autocorrelation functions of an electroencephalogram while the record 
was being taken, thus providing valuable diagnostic information. Some 
digital computers are fast enough to keep up with data from continuous 
tracking experiments, not only keeping a record of time on target, but 
also calculating the autocorrelation of the response and the cross corre- 
lation of the response and the stimulus, to provide immediate informa- 
tion about the characteristics of each subject (see Elkind, 1956). 

The main advantage of on-line calculations is to provide a subject or 
the experimenter with immediate feedback. In training sessions and in 
some experiments, immediate feedback is vitally important. The course 
of training sessions can be determined by the subject’s performance. If a 
subject fails to recognize a stimulus in a recognition experiment, for 
example, the stimulus can be presented again and again until he identi- 
fies it correctly. 


Illustrative example: average response computer. The physiology of 
the nervous system is often studied electrically by stimulating the organism 
and recording the electrical component of the evoked neural response. A very 
clear, precise picture of neural activity can be obtained from a microelectrode 
attached to a single nerve fiber, but when large electrodes are attached to the 
periphery of the cortex, or when a microelectrode intersects several fibers, the 
picture is clouded by background neural activity not correlated with the stim- 
ulus. This electrical “noise” can be overcome by repeating the stimulus many 
times, recording the response to each, and averaging the responses. The con- 
sistent part of the response will maintain its size, while the random backgrounds 
will gradually cancel each other. 

The average response computer (Arc) was built at the Lincoln Laboratory 
at MIT (Clark et al., 1961) to perform on-line continuous averaging of recorded 
clectrical responses. Arc has three built-in programs to do response averaging, 
amplitude histogram compilation, and time-interval histogram compilation. 
While Arc is a special-purpose device, any of the programs could be accom- 
plished with a general-purpose computer that had an analog-to-digital converter 
as an input device. 

The response-averaging program works as follows: The occurrence of a stim- 
ulus signals the computer. After a preset delay of to microseconds, the computer 
starts sampling the continuous voltage being obtained from the recording 
electrode, The computer obtains, from an analog-to-digital converter, a number 
proportional to the instantaneous amplitude of the response voltage. This record 
iv stored in a memory register that has been assigned to the first reading. After 
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a time delay of At microseconds, a second reading is obtained and stored in the 
next memory register. The process continues at intervals of At microseconds 
until the specified number of readings have been stored. Then the computer 
waits for the next stimulus. When it occurs, the computer waits for fo micro- 
seconds and then starts reading the response voltage. The first reading is added 
to the corresponding reading from the first stimulus, and the sum is put back 
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Fig. 8-3. Typical result from Arc. 


in the storage register assigned to the first reading. Similarly, each successive 
reading is added to the corresponding sample from the first stimulus. In this 
way each register accumulates the sum of the values recorded at a certain time 
interval after the onset of the stimulus, The accumulated sums, multiplied by 
a scale factor, are displayed on a cathode-ray tube, allowing the experimenter 
to observe the buildup of the sums, Figure 8-3 shows the emergence of an average 
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response from background activity as the number of averaged responses is 
increased. 


8-9 Clinical diagnosis 


In principle a statistical analysis of medical records could be an immense 
help in making a clinical diagnosis. A computer that had available to it 
a large store of medical records could help if it merely sorted out the 
recorded cases of patients having the symptoms and background char- 
acteristics of the present patient and tabulated the diagnoses and out- 
comes. The computer might, for example, find that 80 per cent of all 
persons who are about ten years of age and have a large swelling under 
the jaw were sure cases of mumps, the others being distributed in other 
categories, such as accidents, swollen glands, etc. Obviously, the phy- 
sician does not need the computer to tell him that his ten-year-old patient 
has mumps, but there might be cases in which the combination of 
symptoms exhibited by a patient is much less clear, and immediate ac- 
cess to a large file of medical case histories would be a boon. The com- 
puter could be more sophisticated, taking into account the relative 
frequency of kinds of various diseases in arriving at an estimated prob- 
ability for each of the possible diseases the patient might have. Ledley 
and Lusted (1959) have discussed these possibilities and suggest that 
the computer can also take account of the possible payoffs or values 
attached to the various diagnoses in order to influence the decisions of 
the computer. They propose that positive and negative values be asso- 
ciated with the expected outcomes of the acceptance or rejection of the 
various treatments, so that the dangers of erroneous treatment can also 
be included as part of the statistical analysis. 

When a large number of symptoms and other patient characteristics 
must be considered, it is sometimes worthwhile to consider a multiple- 
discriminant analysis of the data to provide a simplified decision system 
(Rao, 1952). Each of the variables is treated as a continuous variable 
(1 symptom that is either present or absent can be treated as a variable 
with the values 0 and 1). Multiple-discriminant analysis provides a set 
of new dimensions each of which is a linear combination of the values 
of the original variables. Often a few such factors will be sufficient to 
provide a good categorization of the diseases being examined. 

General problems involved in the automation of medical diagnosis 
are discussed by Smith (1961), Vandenberg (1960), and Ledley (1960). 
An actual case involving the automatic screening of cytological smears 
for cancer, which is more pattern recognition than diagnosis, has been 
described by 'Tolles and Bostrom (1956). 

In practice, a diagnosis is never a simple, single-step affair, Presumably 
several stages of diagnosis or disease selection could be done statistically, 
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It is also conceivable that the best help a computer could give a clinician 
is not to provide a complete diagnosis but to answer specific questions 
about medical history and other medical facts. In this case, we would 
think that a program such as the Baseball program, described in Chap. 
13, might be of more help to the clinician than the statistical-analysis 
routines. Questions phrased in standard English would be typed onto 
the computer’s keyboard or punched into a card for insertion into the 
computer. The computer would translate the question into its own 
terms, consult the stored data, and print or type an answer. 


Problems 


1. What simple trigonometric identity will permit the same subroutine to cal- 
culate both sines and cosines? How can the subroutine be set up so that if 
entered at one location it will compute the sine but if entered at a second it 
will compute the cosine? 

2. In a variant of the psychophysical method of ascending limits, the observer 
is presented with a series of stimuli that gradually increase in intensity, 
starting with a stimulus so weak the observer will never detect it. The trial 
ends when the observer has detected two stimuli. Write a flow diagram of a 
Monte Carlo process that takes as input the number of trials to be run and 
the detection probability associated with each of 20 stimuli (the probability 
of detecting the first stimulus is 0, and the probability of detecting the last 
is 1). The program is to run the specified number of trials and is to report 
the average detection probability on the last stimulus in a trial. How would 
an estimate be made of the variances of this average value? 

3. Plan a Monte Carlo test of some objective method of rotation in factor 
analysis. Specify the factor loadings on three factors for a set of n variables 
arranged to exhibit simple structure. The computer is to obtain factor scores 
for each of N individuals on the three common factors and the n unique 
factors (one per variable) by making n + 3 independent “drawings” from 
a normal distribution with zero mean and unit variance. The computer is 
then to compute the test scores, calculate the intercorrelations, do the factor 
analysis, and apply the rotation method. Suggest some way to evaluate the 
results. 
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Random-number generators 


Mc“ computer applications in the behavioral sciences require 
random numbers. Computer programs for making Monte Carlo 
computations and generating stimuli generally consume random num- 
bers in large quantities. The usual sources of random numbers are printed 
tables, but storing long tables of numbers in the computer is both un- 
economical and inconvenient. In the case of trigonometric functions and 
probability distributions, tables are avoided by having the computer use 
mathematical formula to calculate each particular value that it needs. 
l‘ortunately, several mathematical formulas have now been devised for 
producing sequences of numbers that behave as if they were random. 
With one of these formulas, random numbers can be calculated when 
they are needed. The numbers are called pseudorandom because they are 
obtained from a completely determined calculation. Nevertheless, they 
pass all the appropriate statistical tests of randomness, so that opera- 
tionally they are random numbers. 

In the early days of digital computing, random numbers were obtained 
by several methods that proved to be unreliable. The mid-square process 
wenerated each number by squaring the preceding number and extracting 
the middle digits from the result. Regularities were soon discovered in 
the generated series; worse, after several thousand numbers had been 
generated, the process would tend to generate the number 0 and would 
then continue generating 0 for each succeeding number. In an attempt 
(o correct this failing, the process was modified so that each number was 
formed from the middle digits of the product of the two preceding 
numbers, Unfortunately this process also had a tendency to generate 0 
eventually, Neither of those processes is used today, 
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It is not easy to design arithmetic processes that will generate random 
numbers. Most processes are periodic; they will eventually start gen- 
erating the same sequence of numbers again. The number of different 
numbers that a process will generate before repeating is called the length 
of its period. Some processes will degenerate, repeatedly producing the 
same number or the same short sequence of numbers. Other processes 
will not degenerate, but their total period will be too short. Processes 
that do not degenerate and have long periods may not pass the tests 
of randomness. Randomness is an elusive property; the slightest regular- 
ity will destroy it. Consequently, any new process for generating random 
numbers must be very thoroughly tested to ensure that it does indeed 
pass the tests of randomness. 

Pseudorandom numbers are no more difficult to come by than the 
random numbers printed in statistical tables. Great pains must be taken 
in generating tables of random numbers, as any account of these proc- 
esses will report. The most carefully designed mechanical devices will 
be found to have slight regularities if used to generate millions of digits. 
The RAND Corporation experienced difficulties in generating a million 
random digits by an electronic roulette wheel (Brown, 1951). The elec- 
tronic circuit that serves as the roulette wheel was very carefully de- 
signed and tuned; yet it was found to have very slight biases. Additional 
transformations of the resulting digits were required to produce the 
final unbiased set of digits. It seems unlikely that any further attempts 
will be made to tabulate random digits, since future computations re- 
quiring very large quantities of random digits will certainly be done 
on digital computers. 

Two different schemes are used today for generating random numbers 
by a computational process. In both, the numbers are generated se- 
quentially. Each number is derived in a simple way from the number or 
numbers preceding it. In one scheme, each number is obtained from its 
predecessors by a process of addition; in the other, a multiplication 
process is used. The additive and the multiplicative processes have many 
properties in common. Both can generate large quantities of digits that 
pass the statistical tests of randomness. Both are periodic, in the sense 
that they will eventually start generating the same series all over again, 
but both can be set to generate millions of numbers before the sequence 
repeats. Both are very speedily executed on most computers. Both 
generate numbers that are uniformly distributed in the range from 0 to 1. 
Subroutines for more complex random processes can easily be written 
when a source of uniformly distributed numbers is available. 


9-1 Additive generator 


Pseudorandom numbers can be generated by a very simple addition proc- 
ess, Starting with an initial set of nm random numbers, Xi, Xa...) Xn, 
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that are all between 0 and 1, the procedure generates additional numbers 
X; successively by adding the immediately preceding number to the 
number generated n steps earlier and retaining only the fractional part 
of the sum. In mathematical notation, 


X; = Xj + X jun (mod 1) 


The notation (mod 1) indicates addition modulo 1. In general, addition 
modulo N means that the result is to be the remainder after the sum is 
divided by N and the integral part of the quotient is discarded. Thus, 
addition (mod 1) indicates retention of only the fractional part of the 
sum. On a digital computer, fixed-point numbers can be added, modulo 1, 
by simply ignoring overflow (see Sec. 6-5). Table 9-1 shows a program 


Table 9-1 Subroutine for generating fixed-point random numbers by 
additive process 


Location Content of register Comments 
Program 
ARANDM (STORE A IN RAND1 Save index register 
tLOAD A FROM RAND2 Get location of Xj-1 
LOAD FROM RN:A Get Xj-1 


INDEX A FROM MONE Find location of Xj-n 
JUMPIF A PLUS TO RAND3 If outside the block 


LOAD A FROM N Start over 
RAND3 ADD CONTENT OF RN:A Add Xj-n 
STORE IN RN:A X; replaces X jn 
{STORE A IN RAND2 Save location in block 
TLOAD A FROM RAND1 Restore index register 
JUMP TO 1:A Exit 
Constants 
RAND1 ann Storage for A 
RAND2 0 Location of next RN 
in block 
MONE -1 Constant for indexing 
N N-1 Reset for location 
Data 
RN Block of N random 
numbers 


} If an index register can be set aside for the exclusive use of this subroutine, these 
instructions may be omitted, 


for the additive generator in terma of the hypothetical computer of this 
book, This program yields fixed-point random numbers, If floating-point 
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numbers were needed, a few more instructions would have to be added 
to convert the generated numbers into the floating-point format. 

In common with most computation schemes for generating pseudo- 
random numbers, the additive process is periodic; it will eventually 
repeat itself by regenerating the original n numbers. The period depends 
mainly on n and on the number of digits in the stored numbers. For 
binary numbers with p bits, the period is k, X 2?~1, where k, is an integer 
that depends on n. Table 9-2 shows values of k, for n = 2 to 16. Our 


Table 9-2 k, for n = 2(1)16 


n kn n kn 
2 3 10 889 
3 i 11 2047 
4 15 12 3255 
5 21 13 2905 
6 63 14 11811 
7 127 15 32767 
8 63 16 255 
9 73 


hypothetical computer uses 35 bits to encode a fixed-point number and 
uses the thirty-sixth bit for the sign, so that p = 35. If we use n = 16, 
the period of the additive generator would be 255 X 2%, or about 4 
thousand billion. 

Several empirical tests of numbers generated by the additive process 
are reported by Green, Smith, and Klem (1959). The main results are 
that the generated numbers, for any number of digits and for any n, 
are uniformly distributed and exhibit no significant serial correlation. 
However, the numbers generated by originating sequences with fewer 
than 16 numbers do not pass the runs test, which is a very sensitive test 
of serial dependency. For any n some samples of numbers will pass the 
test, but for values of n less than 16 the runs test was failed by slightly 
too many samples of numbers. The runs test was satisfied for n = 16. 
It seems very likely that for any n greater than 16 the runs test will 
also be satisfied. If alternate numbers are discarded, the additive process 
also passes the runs test for » = 6 and presumably for all larger n. 

A programmer wishing to use the additive process to generate random 
numbers must therefore choose an n of at least 16 or must plan to discard 
alternate numbers. The former is more rapid, and the latter requires 
less storage. Of course, if the problem should happen to use alternate 
numbers for different purposes, then a smaller value of n may be used, 
without discarding alternate numbers. Or if the programmer feels that 
he needs only approximate randomness, as in setting up experimental 
designs, he can ignore the runs difficulty and can choose n according 
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to his convenience. The runs test is very sensitive, and failure to pass 
this test does not mean that the numbers are badly awry. 


9-2 Multiplicative generator 


The multiplicative process for generating random numbers is even simpler 
than the additive process. Each number is generated from its predecessor 
by multiplying that predecessor by a magical constant C, as in the 
following equation: 


X; — CX 5-4 (mod 1) 


Any of a large variety of numbers may be used as the magical constant C. 
Candidates must be integers and must have as many significant digits as 
the word length of the computer. Each potential C must be carefully 
tested, because the particular value of C used will determine not only 
the apparent randomness of the sequence but also its period. Different 
values of C are used for generating decimal and binary numbers. For 
decimal numbers, Moshman (1953) suggests C = 79, 743, or 7!7. For 
binary numbers, Taussky and Todd (1954) suggest a magical constant 
(' = 5**+1, where k is any integer, while Greenberger (1959) suggests a 
magical constant C = 2'§ + 3. The initial starting number should be 
picked from any available source of random numbers. In most cases, 
however, no harm will be done by using an arbitrary starting number. 

The periodic nature of the multiplicative process can be seen easily. 
“ach number in the sequence determines the next number, which in 
turn determines the next, and so on, so that each number indirectly 
determines all the numbers that follow it. If a number ever occurs for a 
second time, the sequence of numbers that followed its first occurrence 
will also follow its second occurrence and so the entire series repeats. 
And sooner or later a number must occur for a second time, because 
there is a limit on how many different numbers can be generated. On a 
binary machine there are only 2% different numbers that can be repre- 
sented by 35 bits, so that a series of 23° + 1 numbers must contain at 
least one duplicate. Of course 2*° is a lot of numbers—over 32 billion— 
and the multiplicative process can be made to produce a sizable fraction 
of them before a repetition occurs. In most problems, the periodicity 
will cause no trouble. 

CGreenberger (1961) and others have made empirical tests of the mul- 
tiplicative generator with C = 2'* + 3, generating 35-bit fixed-point 
numbers. The period of the sequence is 2**, which is more than 8 billion. 
The resulting numbers pass all the tests of randomness, including the 
very sensitive runs test, The magical constant 2! + 3 has the fortunate 
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property that multiplication by it can be achieved rapidly by shifting 
and adding. Multiplying by 2'* is equivalent to shifting left by 18, and 
multiplying by 3 is equivalent to adding the number three times. A 
program for this multiplicative generator is shown in Table 9-3, for the 
hypothetical computer of this book. 


Table 9-3 Subroutine for generating fixed-point random numbers by 
a multiplicative process 





(C = 238 + 3) 
Location Content of register Comments 
Program 
MRANDM LOAD FROM RN Get Xj 
SHIFT LEFT 18 Multiply by 2% 
ADD CONTENT OF RN Xj 
ADD CONTENT OF RN Xj 
ADD CONTENT OF RN Xj 
STORE IN RN (28 + 3)Xj4 
JUMP TO 1:A Exit 
Data 
RN +1376842191+ Last number generated 





+ Initially, register RN must contain a random number with as many significant 
digits as the capacity of the register. 


A variation of the multiplicative method is given by 
X; = CX;.1+ K (mod 1) 


Here both C and K are fixed, odd numbers. Each new number in the 
generated sequence is obtained by multiplying its predecessor by the 
constant C and then adding the constant K. Rotenberg (1960) has 
tested this generation process for C = 27+ 1 and K = 1 for 35-bit 
numbers. The period of this generator is 2**; that is, it generates all the 
numbers that can possibly be represented by 35 bits. However, the fact 
that this period is four times as long as that of the standard multiplica- 
tive process seems essentially unimportant in view of the already tre- 
mendous period enjoyed by that generator. This lengthened period, 
moreover, is achieved at the expense of a slower computation time be- 


cause of an additional computer instruction in the generator subroutine, ~ 


In applications requiring thousands of numbers, time is all-important; 
so the variant is not recommended, 
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9-3 Properties of random number generators 


On most modern computers the multiplicative process of generating 
pseudorandom numbers is faster than the additive process and requires 
less storage. Its period, however, is inherently shorter. If large quantities 
of random numbers are needed, so that the length of the period is an 
issue, then the additive process should be used. Other rare cases may 
arise in which the additive process is better, but for most applications 
the multiplicative process is to be preferred. 

The additive generator requires n random numbers to get started, 
and the multiplicative generator requires one. Obviously, if the same 
starting numbers are used a second time, the same sequence of random 
numbers will be generated. Thus, every new use of the random number 
generator requires new starting numbers. These numbers should be taken 
from random-number tables. Choosing these numbers haphazardly could 
be dangerous, because there is always the chance that, in picking several 
different starting numbers, some bias will occur. 

There is an interesting advantage in using a pseudorandom-number 
generator. Keeping a record of the starting numbers used in an applica- 
tion is equivalent to recording all the random numbers used. Exactly 
identical results could be obtained at a later date just by using the same 
starting numbers. Perhaps many long lists of stimuli have been generated 
and have been used in an experiment, and later some of the stimulus 
lists have been lost. They can easily be recovered by running the genera- 
tion program again, with the same starting numbers in the random- 
number generator. 

Both the additive and multiplicative generators have the property 
that, as more and more of the numbers in a complete period are used, 
slight regularities become noticeable. One such regularity is the uni- 
formity of the distribution of numbers. If more high numbers than low 
numbers happen to be generated early in the sequence, then the latter 
part of the sequence will have more low numbers than high numbers, 
which would not be true of a completely random sequence. In practice 
one should never plan to use more than a small part of the numbers in 
1 complete period. Programs requiring billions of random numbers 
should use the additive generator with its longer period. Even so, the 
prudent user will restart the additive generator at regular intervals, with 
new sets of initial numbers, since different sets of starting numbers are 
virtually guaranteed to give different samples of generated random 
numbers. 

One final caveat applies to both additive and multiplicative gener- 
ators, A single generated number should never be used for more than one 
purpose, [t should never be considered to be a set of random digits. 
Programs should never use some of the digits for one random event and 
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others for another random event. The generated number should always 
be regarded as a random fraction, and one without very many significant 
digits at that. The reason is that the less significant digits tend to exhibit 
uniformities. Empirical studies always find that the rightmost digits are 
too nearly uniform. 

The mechanism behind this empirical fact is easy to find. Both the 
additive and multiplicative generators discard any digits that overflow 
the capacity of a computer word. Suppose that the capacity of a certain 
machine is one digit less than another. Then the sequence generated by 
the smaller machine is exactly that generated by the larger machine 
with its leftmost digit discarded. The sequence with fewer digits will 
have a shorter period. For binary machines its period will be one-half 
that of the larger sequence, because the period of a binary generator is 
proportional to 2”, where p is the number of digits. Thus, the sequence 
of numbers that form one period of the larger machine, when shorn of its 
leftmost digit, becomes exactly two periods of the smaller sequence. That 
is, it becomes two identical series. (In the case of decimal machines, the 
period is proportional to 10”, so that discarding a digit reduces the period 
by a factor of 10 rather than 2.) Extending the same argument, we find 
that, as more and more digits are discarded, the period gets shorter and 
shorter. Thus the digits at the right of the numbers have a short period 
and tend to repeat themselves frequently. Consequently they are more 
evenly distributed than chance would allow. 

Peach (1961) has used this property to argue that these pseudorandom 
numbers can cause serious trouble in studying variability by Monte 
Carlo techniques. However, for the regularities to be troublesome in 
the high-order digits, millions of random numbers would have to be 
involved in a single determination, which is a very rare occurrence. As 
long as the above caveat is obeyed, the generated numbers should give 
no trouble. 


9-4 Other random processes 


The random-number generators that have been presented all produce 
numbers that are uniformly distributed in the interval from 0 to 1. 
We shall henceforth call these numbers random fractions. To obtain 
numbers that have different distributions, further subroutines are re- 
quired. Special routines are also needed for generating more complex 
sets of numbers, such as random permutations and random selections 
from a fixed number of alternatives, either with or without replacement, 
to name just two. Given a source of random fractions, the programmer 


can easily write such routines. In this section, methods will be presented - 


for some of the more common random processes. Other random processes 
are suggested as programming problems for the interested reader, 
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Random normal deviates. Many psychometric problems require 
random samples from a normal distribution. At least two different ways 
have been used for randomly selecting numbers from a normal distri- 
bution. One simple way that works very well is to add several random 
fractions. This technique is based on the central-limit theorem, which 
states that the sum of n independent random variables is distributed 
approximately normally, the distribution approaching normality as n 
tends to infinity. In particular the sum of n random fractions is distrib- 
uted approximately normally, with a mean of n/2, and a variance of 
n/12. It is convenient to choose n = 12, so that the sum will have a 
variance of 1, or n = 24, so that the variance will be 2. With n = 12, 
the approximation is satisfactory for most purposes, and for n = 24, 
the approximation is good enough for Monte Carlo computations. The 
main consideration in choosing n is the extent to which the extremes, or 
“tails,” of the distributions must match. A small 7 will suffice to match 
the major part of the distribution, but the extreme regions, which are 
important in hypothesis testing, are not matched so well with small n. 

The standard procedure for selecting numbers from a nonuniform 
distribution is to select a random fraction and to treat this fraction as a 
percentile in the required distribution. The number corresponding to 
that percentile in the required distribution is then the required selection. 
‘The computer must determine the selected number based on the per- 
centile either from a table of percentage points of the distribution or by 
a mathematical formula from which the percentage points can be 
calculated. In the case of the normal distribution, a formula can be found 
in Hastings (1955), but it is so complicated that storing a table or adding 
random fractions is to be preferred. 

Muller (1959) has compared these methods of generating random 
deviates with a direct method reported by Box and Muller (1958), an 
inverse method reported by Muller (1958), and two other methods. All 
gave satisfactory results with varying speeds and precisions. The in- 
verse method was much faster than the others, but its details are 
intricate. More recently, Marsaglia (1961) reported an even more rapid 
method that is also intricate. Readers contemplating problems that 
require huge quantities of random normal deviates are urged to look 
into these more efficient methods. 

Random selection from equally likely alternatives. To select 
one out of n equally likely alternatives, a random fraction is multiplied 
by n, and the fractional part of the product is discarded. This leaves an 
integer in the range from 0 to n — 1, which may be used to locate the 
particular symbol required, I’or example, the symbols might be stored 
in successive registers starting at register SYMB. To get the selected 
symbol we could place in index register A the integer resulting from the 
above operation and execute the instruction LOAD FROM SYMB;A, 
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Random selection from unequally likely alternatives. If the 
alternatives in a fixed set are to have different probabilities of being 
selected, a portion of the unit interval is assigned to each alternative. 
The size of each portion is proportional to the corresponding probability 
of selection. Then a random fraction is obtained, and the selection is 
made in accordance with the portion of the interval in which the fraction 
occurred. 

For example, three alternatives A, B, C are to have the following 
probabilities: Pa = 0.5, Ps = 0.38, Pc = 0.2. Let R be the random 


fraction. Then: 
If0 < R < 0.4999 --- , A is selected 
If 0.5 < R < 0.7999 --- , B is selected 


If 0.8 < R < 0.9999 --- , C is selected 


Random permutation. The following process will produce a ran- 
dom permutation of n items, Xi, X2,..., Xn. We assume that the items 
are stored sequentially, either on a list or in successive registers. First, 
one of the items is selected at random and put at the head of the list by 
interchanging it with the first item in the list. Then, an item is selected 
at random from the n — 1 items, the first item on the list being excluded; 
this item is placed in the second position on the list by interchanging it 
with the item now in second position on the list. This process is con- 
tinued, by selecting an item at random from the remaining items and 
interchanging this item with the first item among the remaining items 
until no items remain. A detailed outline is as follows: 


_ 


. Set 7 equal to 1. 

2. Choose an integer j at random from the range i <j < "5 e., select 
a random fraction, multiply by n — 7+ 1, add 2, and take the 
integral part of the result. 

. Interchange X; and Xj. 

Add 1 to 7. 

If 7 is less than n, return to step 2; otherwise the process is finished, 


ae 


Random selection without replacement. A subset of m items 
out of n items is to be selected. In effect, the m items are drawn one at & 
time out of the total of n items, the selection being made without 


replacement. Exactly the same procedure can be used here as in the: 


random-permutation process. The only change in the procedure is the 
stop rule: 
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5. If 7 is less than m, return to step 2; otherwise the process is finished 
and the new Xi, X2,..., Xm are the items selected. 


Random selection with marginal constraint. In a certain psy- 
chophysical detection experiment each subject receives 200 trials, on 
100 of which the signal is present and on 100 of which it is absent. The 
presence or absence of a signal on any particular trial is determined at 
random, with the restriction that the signal must occur on exactly half 
the trials. One way to set up the schedule of stimuli is to perform a 
random permutation of a list of 100 A’s and 100 B’s where A stands for 
presence and B for absence. 

The following logically equivalent procedure will produce a random 
ah of n; A’s and n, B’s without requiring an initial list of A’s 
and B’s. 


1. Set n equal to 1; set m equal to ne. 
2. Get a random fraction, R,O < R <1. 
3. a. If R <n/(n +m), record an A, subtract 1 from n, and go to 
step 4. 
b. Hf R>n/(n +m), record a B, subtract 1 from m, and go to 
step 4. 
4. If n + m > 0, return to step 2; otherwise the process is finished. 


Correlated variables. Independent random normal deviates can 
be used to construct random samples from a bivariate normal distri- 
bution with a specified correlation, r. A random selection from this 
distribution is a pair of quantities, say, x; and y;, such that the product- 
moment correlation between x and y approaches r as the size of the 
sample increases. The two correlated quantities 2 and y can be con- 
structed from three uncorrelated quantities a, b, and c by using the 
following formulas, 


a=Vl—rat+vVre 
y=V1l—rb+Vre 


where x, y, a, b, and ¢ are all normally distributed with zero mean and 
unit variance. For each random selection, three random normal deviates 
ure obtained from an appropriate subroutine and are entered as a, b, and 
c in the above equation to produce a and y. 


Problems 


1, Outline a routine to select a symbol at random from five equally likely sym- 
bola, but excluding a particular one of the five symbols, The input to the 
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routine is to be the particular symbol not to be selected, and the output 
of the routine is to be the new selected symbol. 


. Outline a program to generate zero-order approximation to English text by 


selecting at random from the 27 equally likely alternatives composed of the 
26 letters of the alphabet plus space. The text is to have a total of n symbols. 


. Assume that a table is available showing the relative frequency of occurrence 


of each of the 27 symbols in English text: the 26 letters of the alphabet 
plus space. Outline a computer program to generate first-order approximation 
to English text by selecting at random from the 27 symbols, according to 
the relative frequencies specified in the table. (The sum of the relative fre- 
quencies is 1.) A total of n symbols of text is required. 


. Prepare a flow chart of a routine to produce 50 randomized lists of 100 items. 


Each list is to contain 25 of each of four stimuli, labeled A, B, C, and D. 


. The suggested procedure for generating correlated variables required three 


uncorrelated random normal deviates. How can the correlated variables X 
and Y be generated from only two random normal deviates? 

Write a flow diagram for a program to test the output of a process for gen- 
erating random normal deviates. Use this program to determine the fewest 
random fractions that can be added to produce an acceptable random normal 
deviate. 





10 


Stimulus generation 


Bete stimuli for psychological experiments is often a tedious 
task. An experiment on human learning, for example, may require 
many randomized lists of assorted nonsense syllables. An experiment on 
wudition may require a great many different complex waveforms com- 
posed in specified ways to serve as auditory stimuli. Experiments in 
visual pattern recognition sometimes require visual stimuli controlled in 
precise ways to measure thresholds for patterns or forms. Many experi- 
ments in both animal and human behavior include apparatus that is 
controlled by punched tape, punched cards, or specially recorded 
magnetic tape, and some of the experiments use large quantities of 
control tapes. In all cases where the actual production of the stimulus 
materials can be reduced to a routine clerical task, it is possible to 
program the task for a digital computer. With an appropriate output 
mechanism, the computer can generate the stimulus materials or control 
(apes for the experiment. 


10-1 Printed lists 


luvery computer can produce printed lists of information, and many 
psychological experiments have voracious appetites for them. Some 
human-learning experiments may require a different set of randomized 
lists for every subject, so that the total number of lists becomes 
enormous. Psychophysical experiments, studies of human choice be- 
havior, and many others have a need for large numbers of lists, either 
random permutations of items or sets of stimuli that are governed by 
more complex contingencies than simple randomness, Yntema and 
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Mueser (1962), for example, used a computer to generate separate 
stimulus series for each subject in their experiment on human memory 
for changing events. The stimulus material was composed both of facts 
to be remembered and various questions about previously presented 
facts. The computer controlled the relative frequency of occurrence of 
questions and facts and also controlled the proportion of facts that were 
new and the proportion that were repetitions of previous information. 

The lists generated by the computer can be as elaborate as the ex- 
periment requires and as detailed as the patience of the programmer 
permits. Yntema and Mueser wanted the messages in their experiments 
to include only the bare essentials for each fact, such as X RED, 
meaning that the value of the variable x was now the quantity red. The 
lists produced by the computer contained these concise messages, which 
were read aloud to the subjects. But if they had wanted the messages to 
be complete sentences, they could have programmed the computer to 
insert the essential information into a skeleton sentence, producing 
sentences like THE VALUE OF VARIABLE X IS NOW RED. That 
is, the computer can produce precisely what is required for the experi- 
ment, rather than merely producing information from which some person 
can then produce the actual experimental material. 


Illustrative example: stimuli for memory experiment. In a certain 
experiment the subject is required to keep track of the current value of each 
of several variables. The experiment consists of N trials on each of which the 
subject receives a stimulus that is either a message informing him of the current 
value of one of the variables, e.g., “The value of variable X is now red,” or a 
question inquiring about the current value of some variable, e.g., “What is the 
value of variable Y?”’ A message may either change the value of one of the var- 
iables or repeat the present value of a variable. That is, the last message about 
variable X before ‘“The value of variable X is now red” may have reported a 
different value, e.g., blue or exactly the same value—red. The relative proportion 
of changes and repetitions is a dependent variable of the experiment, as is the 
relative proportion of messages and questions. Other dependent variables in- 
clude the number of variables and the number of values that each variable may 
assume. A total of WM lists of stimuli is required for each of several specified 
combinations of parameters. Each list is to consist of N stimuli, preceded by a 
list of the initial values of each variable, to get the experiment started. A flow 
diagram of a computer program to generate the required lists is shown in Fig. 
10-1. 


10-2 Control tapes 


Many experiments in the behavioral sciences involve automatic equip= 
ment for producing and controlling stimuli. Psychophysical experiments 
in audition, for example, generally use special electronic equipment to 
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START 


Read in and preset parameters: 
Q=Probability of question (1-Q = probability of message) 
P=Probability that message changes value (1—P= probability of repeat) 


N;,=Number of variables 

NN, =Number of values per variable 
N,=Number of lists of stimuli 
N,=Number of stimuli per list 


Preset variable counter to zero. 

































Y READ IN 
Read in name of next variable as it is to appear on output list. Preset value counter to zero. 
Read in name of next value as it is to appear on output list. 
Add one to value counter. N,, value names read in yet? ) 
Yes 





Add one to variable counter. N,, variable names read in yet? ) 
[Yes 
Preset list counter to zero. 


Start a new list. Preset stimulus counter to zero. Select an 
initial value at random for each variable. Store these values. 
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Print identifying material and initial values. 


Select a variable at random. 


Select a random fraction R, Is Rless than Q? 
Yes No 


- Stimulus is Question | F----- Stimulus is Message 


Select a random fraction S. Is S less than P? ) 























Yes 
MAKE 


No 
-- Value changes --7 Value repeats LISTS 
Select new value at random. Look up current value 
Store in table of current values. of selected variable, 


Find name associated with 
selected variable and value; 
Insert in message format: 

"THE VALUE OF __ 1S NOW___." 











Find name associated with selected 


variable; Insert in question format: 
"WHAT IS THE VALUE OF __ ?" 





Add one to stimulus counter, Have N, stimuli been printed? 





No 
Yes 
No Add one to list counter, Have N, lists been printed? 
es 
sTop 


Fig. 10-1, Flow chart of routine to prepare list for memory experiment, 
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produce the auditory signals for the observer, and this equipment is 
generally controlled automatically by means of punched tape, punched 
cards, or some similar digital device. A general-purpose computer cannot 
efficiently usurp the functions of the special apparatus, but a computer 
can still be useful if large numbers of control tapes are required. A 
computer fitted with the appropriate tape punch can be programmed 
to punch the required control tapes for any specified sequence of stimuli. 
Even without the appropriate punching mechanism, a computer might 
be useful in preparing lists from which clerks could prepare tapes, but 
automatic punching is much preferable. 

Programming for control-tape production is generally trivial, except 
for the actual output processing, because the experiments for which the 
tapes are being made generally have a simple design. A psychophysical 
experiment in auditory signal detection, for example, might use a two- 
alternative forced-choice procedure and might require 200 stimulus 
presentations in an experimental session. The tape for that session must 
contain 100 of each of two punch codes, in a random order, and we may 
simply make use of a subroutine for random selection with marginal 
constraint. The specifications for the tape become more complicated if 
a few “catch” trials—trials on which no signal is presented—must be 
interspersed within the normal stimulus sequence. Other experiments 
may involve more than two alternatives in more elaborate design, but 
whenever the specifications for the control tapes can be set down 
explicitly, a computer can produce them. 


10-3 A little foresight 


The computer program that generates lists for an experiment can be 
helpful in other ways, if it has been prepared with a little foresight. If 
the program allows the parameters that specify the nature of the lists 
to be changed easily, then it can be used to generate a variety of lists for 
pretesting and exploratory work before the final parameters are chosen 
for the experiment proper. Often, in planning a new experiment, the 
scientist will not know how widely to space the values of his independent 
variables so that their major effects will be clear in the experimental 
results. Many exploratory studies have to precede the formal experi- 
ment, and the computer can facilitate these studies if the needed lists 
can be easily obtained. The same characteristic of the program—freedom 
in changing parameters—may also be helpful in producing material 
for follow-up studies that are suggested by the results of the main 
experiment. 

A computer program for producing stimuli can also, with a little 
foresight, produce materials for use in analyzing the results, If the results 
are to be tabulated by hand, this may involve no more than preparing 
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a few additional lists. In the memory experiment illustrated above, for 
example, the correct answer could have been printed at the side of each 
question. If the results are to be analyzed by the computer, then it must 
keep a record for itself of the stimuli that it produced. By planning the 
analysis before the experiment is run, the experimenter can program the 
computer to prepare for the analysis at the same time that it is preparing 
for the experiment. 

The most straightforward way to record the stimulus series is for the 
computer to list the series on magnetic tape, punched cards, or punched 
tape, in a form that can easily be read back into the computer. Each 
recorded series must of course be accompanied by some appropriate 
identifying material, so that the record of the subject’s responses can be 
matched with the appropriate stimulus series. 

If, either by desire or lack of foresight, no record has been kept of the 
generated series, the computer can still be helpful in analyzing the 
results. If the stimulus-generating program used a pseudorandom- 
number generator, or if no random numbers were involved, then the 
program can be set to generate exactly the same stimuli again. This time, 
however, instead of producing the stimulus it can be set to produce an 
input to a data-analysis program. Often, rerunning the generation 
program will be more economical than saving lists. 


10-4 Auditory stimuli 


A computer is not necessarily limited to the production of printed lists 
for experiments. It can be programmed to produce any sort of output 
that its output devices permit. One possibility is to connect a loud- 
speaker to a particular bit in the accumulator or some other particular 
register of the machine. The rate at which this bit is changed back and 
forth from 1 to 0 governs the frequency of the output tone of the loud- 
speaker. The computer can then produce recordings on standard tape 
recorders of any preprogrammed sequence of tones. By using several 
different bits and mixing the results it is possible to generate chords and 
more complicated musical patterns. Computers can play complicated 
music, 

Historically, loudspeakers were attached to computers, not to play 
tunes, but rather to monitor programs. Every computer program will 
produce its own characteristic growls, flutters, crackles, and even Bronx 
cheers. While programs are seldom sufficiently regular to make tones, a 
program in a short loop will produce a tone whose frequency depends on 
the length of the loop, So if the computer squeals when a program is 
being checked out, the programmer knows that his program is in a short 
loop. It is remarkable how quickly one can learn to identify various 
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parts of a program by the sounds they make and how quickly one can 
hear that something has gone wrong. 

For an experiment in auditory pattern recognition, White (1960) used 
a loudspeaker attached to a computer to generate distortions of familiar 
tunes, such as “Old Black Joe,” “Swanee River,’ “Yankee Doodle,” 
etc. The tonal patterns of these tunes were altered in various specified 
ways in an attempt to determine what properties of the tonal pattern 
were most important in their recognition. The computer program re- 
ceived as input the succession of tones for a correct version of a par- 
ticular tune and a set of rules governing the way in which the tones were 
to be altered before being presented. For example, all tonal intervals 
could be doubled, or all notes could be the same tone, etc. The computer 
could alter the temporal relationships as well as the tonal relationships; it 
could, for example, eliminate the rhythm completely. 

To produce more complex tones than the simple fundamental frequency 
oscillations mentioned above, it is necessary to use a digital-to-analog 
converter. In this case the computer must determine the amplitude of 
the waveform at each instant in time and send this amplitude infor- 
mation to the digital-to-analog converter, where the amplitude is con- 
verted from a digital number to a voltage and recorded on analog mag- 
netic tape. More specifically the computer calculates the amplitude of the 
required waveform for each of many very small time intervals as, for 
example, 1 millisecond. The length of the time intervals depends on the 
highest frequency that is to be present in the output waveform. If 
the computer cannot, in fact, calculate the succession of amplitudes 
sufficiently rapidly for the time intervals required, the tape must be 
recorded at a slower speed than it is played. The computer must then 
alter the calculated frequencies to compensate for the speed difference, 
Care must be taken that the relative intensities of the various frequencies 
are not altered. 

Illustrative example: tone generator. A loudspeaker is connected to 
the least significant (rightmost) bit position in the accumulator. When this bit 
is a 1, there is a certain positive voltage across the speaker; when the bit is 0, 
there is no voltage across the speaker. Thus the input to the speaker is perfectly 
correlated with the value of the bit. The fundamental frequency of the resulting 
tone is equal to the number of cycles per second, where a cycle is a change from 
1 to 0 and back to 1 again. This is half the frequency with which the bit is 
changed. 

The program segment in Table 10-1 will play a tone of a certain frequency, 
controlled by FREQ, and for a certain duration, controlled by DUR. The fre+ 
quency of the tone is half the frequency with which the add instruction is 
executed. Each of the two instructions in the WAIT loop is executed F + 1 
times; so the add instruction is executed once every 2F + 6 instructions. If the 
computer executes 8 instructions per second, the add instruction is executed 
8/(2K + 6) times per second, and the frequency of the tone is 8/(4F + 12) 
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Table 10-1 Program for tone generator 


Location Content of register Comments 
Program 
LOAD A FROM DUR 
PLAY ADD CONTENT OF ONE Change tone bit 
LOAD B FROM FREQ | Wait before changing again 
WAIT INDEX B FROM MONE | Count down F 
JUMP IF B PLUS TO WAIT 
INDEX A FROM MONE | Done waiting 


JUMP IF A PLUS TO PLAY | Test if tone finished 
(Next part of program) 


Constants 
MONE -1 Minus 1, for indexing 
ONE +1 For bit changing 
DUR D Duration control 
FREQ F Frequency control 


cycles per second. For example, if S = 100000 and a tone of 500 cycles per 
second is wanted, F must be 47. 

Similarly the duration of the tone is (D + 1)(2F + 6) instructions, or (D + 1) 
(2F + 6)/S seconds. A computer having § = 100000 requires F = 47, D = 199 
to produce a 500-cycle tone with a duration of approximately 4% second. 


10-5 Visual patterns 


Stimuli for research on visual recognition can be produced by computers 
that have as one of their output devices a cathode-ray tube (CRT), 
which is a display tube similar to a TV picture tube. These computers 
can be programmed to display any complex pattern of dots and lines 
on the CRT. By means of an automatic camera, the computer can 
automatically record these displays on film. The resulting filmstrip can 
then be used as the stimulus material for experiments in visual perception 
(see Green, 1956, 1961). 

The display surface of the CRT can be thought of as a piece of graph 
paper with X and Y coordinates. The basic computer instruction as- 
sociated with the CRT is, in effect, DISPLAY POINT AT X, Y. 
When this instruction is given, a point at the specified coordinate po- 
sition (X,Y) is displayed for a brief time, e.g., 25 microseconds. The 
precise way in which the X and Y coordinates are set up for the CRT 
varies from one computer to the next; but in every case, a pair of co- 
ordinates must be specified for each point to be displayed. The order in 
which points are displayed is immaterial, There is no built-in scanning 
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mechanism similar to a TV scan. Instead, the X and Y coordinates of 
each dot in the picture must be computed; the dots can be displayed in 
any convenient order. 

The automatic camera is set up facing the CRT, and the space be- 
tween them is shielded with a hood. The camera shutter remains open 
while the computer is making the entire display. Thus the film is ex- 
posed to each point that the computer has displayed since the shutter 
was opened. At the end of the display a separate computer instruction 
CHANGE FILM FRAME closes the shutter and advances the film 
to the next frame. The shutter then opens again, and a signal is sent 











Computer CRT and Hood Camera 
associated circuits 


Fig. 10-2. Sketch of physical arrangement of CRT and camera on a computer. 


back to the computer that the camera is ready for a new picture. Figure 
10-2 shows the physical arrangement of the CRT and the camera. 


Illustrative example: drawing a straight line on a CRT. To display a 
straight line on a CRT that can be instructed only to paint dots we must gen- 
erate a sequence of closely spaced dots, from one end of the line segment to the 
other. The routine charted in Fig. 10-3 is based on the parametric equations 
of a straight line. Given two points on the line, (Xi, ¥1) and (X2,¥2), the para- 
metric equations are 


X = X,+ K(X. — Xi) 
You ¥i+K(¥2 — Vi) 
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As the parameter K varies from minus infinity to plus infinity, the point (X,Y) 
traces the entire line. Note that, for K = 0, (X,Y) = (X1,Y) and, for K = 1, 
(X,Y) = (X2,Y:2). For values of K between 0 and 1 the generated points will 
lie on the line segment between the two given points. Thus, if we preset K 
to 0 and gradually increase it to 1, using the equations to compute a point for 
each value of K, we shall generate a series of points on the required line segment. 

One problem remains: how closely to space the dots. Assume that we know 
from experience that the appearance of a smooth line can be achieved if the 
distance between successive dots is A. According to the pythagorean theorem, 
the distance between the given points is 


Distance = V (Xe — X))?+ (Y2— Y1)? 


Then N, the number of dots to be generated, is obtained by dividing the dis- 
tance by A and rounding to the nearest integer. To generate the successive 
points on the line, K should be in- 
creased from 0 to 1 in steps of 1/N. ENTER 

The flow chart in Fig. 10-3 shows the 
routine to display a line segment be- 
tween points (X1,Yi1) and (X2,Y2). Fol- 
lowing the presets, N is computed, 
as explained above. Next, since the 
amounts to be added to each coordi- 
nate in each step are constants, they 
can be computed before the main loop 
is entered, as shown in the third box. 
‘Then the display loop is entered to 
display the initial point (X1,Y1). The 
loop cycles through its three steps, 
display, count, and increment, until 
COUNT exceeds N. On the last cycle, 
COUNT will equal N, and the point 
(Vo, ¥2) will be displayed; then the loop 
will terminate. 















Store X,, Y;, Xo, Yo. 
Preset X=X,, Y=Yj, count =0 






Compute number of steps: 


N= ¥(X2- X1)? + (Yo - Y,)? 


A 


Compute step increments: 
DX =(X_ - X,)/N 
DY =(Y2- Y,)/N 


Increment point: 







The most prevalent use of CRT 
displays is in graphing the results 


Add DX to X 
Add DY to Y 
ee 
of computations, as illustrated in 


Add one to count; 
Is count greater than N? 
lig. 10-4. The computer not only Yes 


plots the results but also draws the EXIT 
raph and writes the captions. The 
pet ni-numnt ‘ ae ti Fig. 10-3. Flow chart of routine to 
NOIB & umpers in whe capuon display a straight-line segment be- 
are formed by patterns of closely — tween (X,Y;) and (X:,Y2). 
spaced dots, Special subroutines are 
available for generating the dot patterns, and more complex sub- 
routines can be used to generate an entire graph, complete with cap- 
tions, 
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Psychologists have found several other uses for computer-generated 
CRT displays. Green, Wolf, and White (1959) used dot patterns to 
study the visual detection of statistically formed patterns. They defined 
a square array of dct positions—usually 128 X 128 dot positions—and 
formed patterns by displaying dots at selected positions in the array. 
Statistical control of the pattern’s detectability was achieved by omit- 
ting some dots from the figure and adding some dots not in the figure. 
For example, a pattern of eight horizontal bars was formed by defining 





Fig. 10-4. Typical graphical output from CRT. 


a figure of four bars consisting of rows 1 to 16, 33 to 48, 65 to 80, and 
97 to 112 in the 128-row display. The other rows were considered back- 
ground. For a particular display, probability p. was assigned to the 
figure and probability p, to the background. Each dot in the figure had 
probability p: of appearing in the display. When p, = 1.00 and p2 = 0, 
an undisturbed pattern of bars was produced, but when p; was not very 
different from 2, the pattern was barely discernible. A sample bar 
pattern is shown in Fig. 10-5. 

The computer program used by Green, Wolf, and White to generate 
bar patterns had several adjustable parameters, In addition to p; and 
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p2, adjustments could be made in the number of bars, the number of 
rows and columns in the dot array, and the dot spacing. For each 
selection of parameters, the computer generated a series of 100 displays, 
50 with horizontal bars and 50 with vertical bars, in random order, to 
serve as stimuli in a forced-choice psychophysical experiment. 

There are many other possibilities. Other geometric figures can be 
embedded in dot matrices; Fig. 10-6 shows a small square within a larger 
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Fig. 10-5. Pattern of eight horizontal bars. 


square. Julesz (1962) has used a variety of statistical rules for controlling 
patterns and has made some very interesting displays. 

White (1960) has used dot figures in a different way; in his displays 
all the dots in the figure are present, and there are no background dots. 
He produces a sequence of pictures with the figure changing on successive 
frames. Between successive frames, all of the dots in the figure undergo a 
two-dimensional random walk, That is, the computer makes a small 
random change in the X and Y coordinates of each dot. In this way, 
cach dot gradually moves away from its initial position, and the figure 
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gradually dissolves into chaos. Several different stages in the disinte- 
gration of a figure of the numeral 5 are shown in Fig. 10-7. These films 
are used in studies of form recognition. 

Still another perceptual phenomenon, the kinetic depth effect, has 
been studied by using computer-generated displays. The kinetic depth 
effect occurs when the shadow of a moving three-dimensional object is 
seen to have depth. Wallach and O’Connell (1953) and Wallach, 
O’Connell, and Neisser (1953) studied this effect with a shadow caster. 
A wire figure is held between a source of light and a translucent screen; 





Fig. 10-6. Small square within a large square. 


the observer views the resulting shadow on the other side of the screen. 
If the figure is irregular and unfamiliar, its shadow is generally seen as 
two-dimensional (flat), but when the figure is rotated about its vertical 
axis, observers generally see its shadow in 3-D. After experiencing this 
effect, observers may see the stationary figure in 3-D. 

The shadow caster limits the kind of displays that can be used in 
studying the kinetic depth effect. A computer can make the same kind 
of display on a CRT by calculating the position of the shadow of a 
specified rotating form. Both the figures and the rotation are specified 
in terms of mathematical equations. The computer is limited only by the 
programmer’s ability to specify figures and rotations, 
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Green (1960) used a computer to make motion pictures showing the 
kinetic depth effect. The motion pictures are made one frame at a time, 
animation-style. Each frame shows the 2-D projection (shadow) of the 
figure at a particular instant in its rotation. Between successive frames 
the computer obtains the new position of the 3-D figure by calculating 
the X, Y, and Z coordinates of its points, corresponding to a movement 
14) second in duration, with the specified rotation. The computer then 
calculates the 2-D projection of the figure. The outline of the computer 
program is shown in Fig. 10-8. Note that, as soon as the display has been 
finished and the impulse given to the camera to change the film frame, 
the computer is free to continue its calculations so long as it does not 
have to display another point. Should the computer come upon a display 


Computer Generated Displays 






Gradual dissolution of a pattern 


by a two-dimensional random walk 
Fig. 10-7. 


instruction while the camera is advancing, the program is simply held 
up until the film is in position in the camera. If the program is arranged 
properly, time that would otherwise be wasted in the film advance cycle 
can be used to do the computations. This is a significant saving, since 
it may take 4 to 4 second to advance the film. In terms of high-speed 
digital computers this means avery considerable amount of computation 
time, on the order of 2,000 to 10,000 instructions. 

A cathode-ray tube on the computer is not absolutely necessary in 
order for the computer to generate pictorial outputs. A standard printer 
can be used to generate pictures by cleverly arranging the printed 
characters. The printed characters on the page are used in the same way 
as the dots in a dot array on the CRT, This technique is limited, but it 
is adequate for some experiments that require pictorial information, 
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Other output devices can also be used. For example, automatic data 
plotters are available at some computer installations; either these are 
attached to the computer directly and draw graphs or plot points on a 
graph at the instruction of the computer, or they are off-line devices 
that are controlled by a data tape or a deck of data cards, each specifying 


START 









Read in parameters specifying figure, rotation, 
number of frames Preset figure. Preset type 
and speed of rotation. Preset frame counter. 


Display dots in figure. 
Display lines in figure. 








Advance camera 





Rotate points in 3-D 





Project points onto 2-D 


Advance frame counter; 
Have enough frames been made? 
Yes 
STOP 


Fig. 10-8. Flow diagram of 3-D display program. 


a certain character and the X, Y coordinates of the position of this 
character on the display. 


10-6 Computer-controlled experiments 


The most elaborate role for a computer in experimentation would be as - 


an active participant. The computer might serve as the experimental 
apparatus itself or as a control apparatus for the experiment, It can 
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generate the control signals or in some equivalent way produce the 
stimulus to the observer and then record the response of the observer. 
It might be programmed to react to the response in contingent ways so 
that the sequence of stimuli that it produces for the subject depends 
upon the past responses made by the subject. 

Small special-purpose computers have often been used for experi- 
mental setups, but general-purpose computers are usually thought to be 
too expensive for psychological experiments. The rate of information 
produced in experiments with organisms is much smaller than the com- 
putational capacity of the machine, so that such an application would be 
a very inefficient use for a computer. There is a good chance that in the 
future computers can be used directly in experiments by means of some 
time-sharing device. When the computer is not required in the experi- 
ment, i.e., while the subject is deciding which button to press, it can be 
working on some other problem. At present, some computers have the 
facility for time sharing, but very little use has been made of this. 

Smith (1961) has recently proposed several variants of standard 
psychophysical procedures, wherein the stimulus to be presented to the 
subject on a given trial is determined on the basis of the subject’s re- 
sponses in previous trials. For example, in the usual up-and-down 
(staircase) method a no response causes the next more intense stimulus 
to be presented, a yes response the next less intense stimulus. Smith 
points out that the up-and-down method could be modified by decreas- 
ing the intensity of the stimulus by three steps for each detection, while 
still increasing it by one step after each miss. This would tend to concen- 
trate observations in the neighborhood of the 25th percentile. Many 
other modifications are possible; each will tend to concentrate obser- 
vations in a certain part of the psychometric function. While the 
experimenter can keep a paper-and pencil record of what happened in 
order to choose the next stimulus in one of Smith’s complicated pro- 
cedures, it would clearly be easier if an automatic system such as the 
digital computer were available to keep track of the previous results and 
choose the next stimulus. 

The stimulus-generating ability of computers has been combined with 
the control possibilities by Swets (1961), who programmed a computer to 
run an experiment in auditory recognition. The computer generates the 
complex auditory patterns that the subject is to identify, using the 
techniques described earlier in Sec. 10-3. The subject is seated before a 
typewriter connected directly with the computer. The computer types 
& message indicating that the experiment is about to begin and listing 
the symbols that the subject is to use to identify the various stimuli. 
Then a particular stimulus is presented, The subject makes a guess as 
to which stimulus it was and types the appropriate symbol. If he is 
correct, the computer proceeds to generate the next stimulus, If he is 
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wrong, however, the computer displays the stimulus corresponding to 
the subject’s choice and repeats the stimulus presented on that trial. 
The subject can compare the two and see his error and must then guess 
again. The computer is so fast and versatile that two subjects can be 
run in this experiment at the same time. The two subjects will be making 
different responses and thus will be running asynchronously. The com- 
puter can manage to keep them both occupied at the usual rates of 
responding. 

Some experiments are so elaborate and demanding that they cannot 
be done without a computer. Experiments on man-computer control 
systems, described in more detail in Chap. 12, require the computer to 
spend all its computation time devising stimuli for the subjects in the 
experiment (Rowen, 1951.) The stimuli are not filmed in advance but 
are generated and presented to the operators on direct-viewing CRTs 
during the experiment. The operator communicates with the computer 
by push buttons, keyboards, and pointing devices. A man-computer 
system generally has several operators, who perform different functions 
but who must work simultaneously and cooperatively. The computer 
must generate displays for all the operators and must sense the inputs 
made by each of them. Since the operators’ actions are supposed to 
affect the future course of events in the experiment, the computer must 
make corresponding adjustments in the generated stimuli. Finally, the 
computer must make a record of the operators’ actions and the actual 
stimuli generated, for later analysis. 


10-7 Technical details of CRT displays 


A standard CRT produces a display by pointing a focused beam of 
electrons on the display surface, which is coated with a phosphorescent 
material that glows when bombarded by electrons. In a television set the 
beam is automatically swept back and forth across the surface, ‘“‘paint- 
ing” a series of adjacent lines. The strength of the beam is modulated by 
the incoming TV signal to produce gradients of brightness, so that a 
picture is formed. But there is no automatic sweep on a CRT connected 
with a computer. The DISPLAY command causes the circuits to point 
the beam at a certain spot and then to turn on the beam for a brief in- 
stant. The beam is normally off and is turned on only when it is in 
position, and only long enough to paint a dot. 

At most installations, letters and numbers are produced on the 
display by programming a succession of dots, but special character- 
generating devices are also available. The Charactron (Harris et al., 
1956) produces characters by passing the defocused electron beam 
through a metal stencil, thus shaping the beam to the required character. 
The shaped beam is then deflected to the required place on the display, 
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The Calliscope (Perry and Aho, 1959) is based on an elaboration of 
Lissajous figures. The results of a two-dimensional Fourier analysis of 
the required figure are used to generate voltages that cause the focused 
beam to trace the figure. 

The brightness of a display on a CRT depends on the characteristic 
way in which the tube’s phosphor responds to the energy in the electron 
beam; the energy in turn depends on the voltage and current in the beam 
and on the duration for which the beam is energized. The brightness of 
television pictures is controlled continuously by the voltage of the beam. 
Some computer-controlled CRTs allow several discrete levels of bright- 
ness to be programmed. On others, there is only one permissible level, 
but the brightness of a spot can be increased by displaying it several 
times in quick succession, thus in effect increasing the duration of the 
beam on the dot. 

The phosphor on a CRT display continues to glow after the electron 
beam has stopped bombarding it. The brightness of the afterglow decays 
rapidly, but the rate of decay depends on the phosphor. Some phosphors, 
such as P1, have a very short persistence, and are essentially instan- 
taneous. Others, like P7, P14, and P19, have a long persistence, the 
image remaining for many seconds. New tubes with additional control 
mechanisms are being developed to allow the painted image to remain 
bright until it is “erased” by a separate control signal. 

Direct-view displays. For some applications, a CRT display is 
viewed directly by a man, rather than being photographed for later 
film projection. Man-computer systems often involve many human 
operators watching computer-generated CRT displays. A CRT that: is 
to be viewed directly by an operator should, if possible, display a bright, 
steady image. Special controlled persistence tubes have been developed 
for this purpose, but they are not completely satisfactory for many 
applications. Instead, bright, steady displays must be achieved by using 
a short-persistence phosphor and regenerating the display at a rate above 
the flicker fusion frequency for the display. Since fast display rates are 
costly, the slowest rate compatible with an apparently steady image is 
desired. Mitchell (1961) has made a careful analysis of this problem. 

In many applications there is not enough computing power to repaint 
the display many times per second. Here, a long-persistence phosphor 
must be used, and the afterglow must be depended on to display the 
information to the operator. Choosing the best phosphor can be difficult. 
If the image decays too slowly, the old trace will persist after a new 
display is generated and will tend to mask the new information. But if 
the image decays too rapidly, then either there will be moments when 
no display is visible or the computer will be required to regenerate the 
displays too often, 

Pointers, When a CRT is used as a communication device in a 
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man-computer system, some means must be provided for the operator 
to indicate an item on the display to the computer. Several devices 
allow the operator to control the position of the electron beam in the 
tube. When the beam has been positioned over the right spot, he pushes 
a button, causing the positioning voltages to be read into the computer 
to indicate the coordinates of the spot. 

A more efficient pointing device that has become very popular is the 
light gun. A light gun is a photoelectric pointer that makes use of the fact 
that successive characters on a CRT display are generated at different 
times. The operator aims the gun at his target and pulls the trigger, 
activating a photocell. When the target is next displayed, the light from 
the displayed target registers on the photocell, which signals the com- 
puter that the target just displayed is the one the operator means to 
indicate. To aid the operator in pointing, some guns have a built-in 
light beam that is colored so that the photocell is insensitive to it. This 
beam is projected through the optical system of the gun onto the display, 
showing just where the gun is pointed. Other guns are built with photo- 
transistors and are so small that they are better called light pencils. A 
pencil needs no light beam, since it can be put down directly on the re- 
quired spot on the display. Many other pointing devices based on similar 
principles can be used. 


Filming. Generally the CRT associated with the camera is a 5-in. flat- 
faced tube with a P1 phosphor. This phosphor has no persistence; the displayed 
dot is visible only when the electron beam is hitting the phosphor. For this 
reason a larger CRT with a persistent phosphor is often provided for visual 
monitoring. 

Brightness control is difficult on computer-generated films. The brightness 
of the CRT is not easily kept constant from day to day, because slight voltage 
changes can make visible differences in brightness. Also, uniformity in film 
processing is difficult to achieve—slight differences in the temperature of the 
developer or the length of time the film remains in the developer will cause 
visible brightness changes. It is good practice to take all the film necessary for 
one experiment at one computer run and to have it all developed in the same 
batch. Even with these precautions, precise control of brightness is virtually 
impossible. 

A variety of cameras may be used, depending on the intended use of the 
film. If each picture is to be viewed and/or printed separately, the only require- 
ment is a clear image. If the film is to be viewed or projected in a filmstrip 
projector, the spacing of successive frames on the film becomes important. If 
the film is to be projected in a motion-picture projector, the spacing must be 
controlled very accurately, to ensure accurate frame-to-frame registration. Gen- 
erally the automatic camera attached to a computer uses 35-mm film and has a 
detachable film magazine with a capacity of at least 100 ft of film, Some 35-mm 
cameras, such as the Beattie camera that is standard equipment on the CRT 
attached to the IBM 704/709 computers, make no provisions for controlling 
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the spacing of successive frames. Others, such as the Fairchild 0-15 and the 
Traid, provide accurate single-frame spacing and use a registration pin in the 
film sprocket holes to control the registration of each frame. For motion-picture 
work, a 16-mm camera is generally preferable, since 16-mm motion-picture 
projectors are much more accessible than 35-mm projectors. Of course, photo- 
graphic laboratories can reduce 35-mm film to 16-mm film by rephotography 
but this is a cumbersome arrangement. 

When many pictures are being taken, the amount of time that the camera 
spends in changing the film frame becomes a significant factor. The computer 
is typically designed so that calculations can proceed while the film is advancing. 
The interlock arrests the computer if a display instruction is given while the 
film is in transit. In many cases, the amount of calculation between successive 
pictures consumes only a fraction of a second, so that most of the running time 
of the computer program is consumed in indexing the camera. The Beattie 
camera uses about 3% second to advance the film. The Fairchild and Traid 
cameras use about 4 second, and 16-mm motion-picture cameras can be set up 
to use less than Mo second for film advance. 


Problems 


1. For an experiment on the redundancy of English text (Miller and Friedman, 
1957) errors must be inserted in a 200-letter excerpt. Each of the 27 characters 
(26 letters and space) has a probability p of being altered. Three different 
kinds of alteration are required: blanking, in which the altered character 
is replaced by a blank, garbling, in which the altered character is replaced by 
a different randomly chosen character, and deletion, in which each of the 
following characters is moved up one place, shortening the text by one char- 
acter. Write a flow diagram for a program that receives N characters of text, 
together with p and the type of alteration wanted, and that produces the 
altered text. 

2. A staccato note is a tone of a certain duration followed by a silence of about 
the same duration. Write a program to generate a staccato note, given a 
parameter specifying the frequency of the tone and a parameter specifying 
the duration of the total interval, tone plus silence. Indicate the relation 
between the frequency parameter and the actual frequency of the tones and 
between the duration parameter and the actual duration of the interval. 
The loudspeaker may be assumed to be connected with any convenient bit. 

3. A trill is a rapid alternation between tones of two neighboring frequencies. 
Write a program that will produce a trill of a specified total duration, at a 
fixed rate of alteration, between two tones of specified frequency. 

4. Any letter or numeral can be made fairly well on a CRT by displaying 
appropriate dots in a 7 X 5 matrix, e.g.: 


oo 0@0 
0oo@e00 
oeoo090 
e@eeeeo 
eoooe 
@oooe 
oeeeond 
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vw 


71 


Since our hypothetical computer has more than 35 bits in each word, the 
pattern of bits—whether 0 or 1—in 35 of the bit positions can be made to 
correspond with the 35 dots in the 7 X 5 matrix. 

Write a subroutine, either as a flow chart or as a symbolic program, to dis- 
play a character controlled by the bits in a given storage register, the lower 
left corner of the character being positioned at (Xo,Yo) and the dot spacing 
being A in both directions. 


. The gestalt organizing principle of common fate can be demonstrated by 


moving a configuration of randomly placed dots in a visual field containing 
a second stationary configuration of random dots. Prepare a detailed flow 
diagram of a program to generate a motion-picture demonstration for this 
phenomenon. 


. One way to specify a psychophysical threshold for the perception of form 


is to add dots to a figure, one by one, starting from a single dot, until the 
figure is recognized. Write a flow chart for a program to generate filmstrips 
for such an experiment. The input to the program is to be N, the number of 
dots in the figure, and N pairs of coordinates (X;,Y;),7 = 1,..., N, repre- 
senting the dots composing the figure. On the first frame of the film, one 
dot is to be selected at random and displayed. On the next frame that 
dot and another randomly selected dot are to be displayed. Another dot is 
to be added at random on each successive frame; the Nth frame will show 
the complete set of dots. 


. The parametric equations of a circle of radius R with its center at (Xo, Yo) are 


NX = Xyo+ Reos 0 
Y = Y¥o+ Rsin @ 


The parameter is 6, which varies from 0 to 360°. The dot spacing is related 
to the increment in angle L6, by the approximation 


D6 = arcsin 


R 


Assuming the existence of subroutines to compute sin, cos, and aresin, write 
a flow chart for a routine to draw a circle of radius R with its center at Xo, Yo. 
The parametric equations of an ellipse are 

X = Xo + A cos (6 — 6) 

Y = Yo + Bsin (@ — 61) 


where (Xo, Yo) is the center of the ellipse, A is half the length of the major 
axis of the ellipse, B is half the length of the minor axis of the ellipse, and 0, 
is the angle between the X axis and the major axis of the ellipse. The param- 
eter is 0, which varies from 0 to 860°. The widest spacing between dots occurs 





Stimulus generation 195 


when @ — 6; = 90°; the increment in angle, Dé, at that point is given by 
_ A 
D6 = = 
aresin 5 


where A is the spacing and D6 is the increment in the angle 6 — 4). 
Given Xo, Yo, A, B, 6;, and A, and assuming the existence of routines for sin, 
cos, and arcsin, write a flow chart for a routine to draw an ellipse. 








11 


Computer models of psychological processes 


HE flow chart of a digital-computer program is a convenient device 

for depicting complex computational processes. Mathematical mod- 
els that include many contingent branches and iterations can best be 
understood with a diagram. Flow charts are also convenient for describ- 
ing processes that are not mathematically complex but that involve 
intricate information handling, such as business accounting, airline 
reservations, military logistics, and the like. Quite naturally, then, some 
psychologists have begun to use flow charts to express models of psy- 
chological processes. 

A model based on a flow chart may be called an information processing 
model, to distinguish it from a mathematical model. Flow charts are 
much more flexible than mathematical equations and are particularly 
useful in modeling the so-called “higher” mental processes, such as 
problem-solving, where a complex kind of feedback arrangement must 
be pictured. Miller, Galanter, and Pribram (1960) have written a de- 
lightful, stimulating essay espousing information processing models and 
suggesting how they might be applied in many areas of psychology. 
They introduce a special symbol for a simple feedback loop, which they 
call a Tore unit, meaning Test-Operate-TEst. In computer terms, a 
Tore unit is a program loop—a DO loop in Fortran. The Torr unit 
has a test phase that detects the presence or absence of a certain condi- 
tion and an operation phase that carries out some action if the condition 
is not met. For example, the test in the process of hammering a nail into a 
board determines whether or not the head of the nail is in contact with 
the board, while the operation hits the nail on the head with a hammer, 

A flow chart is often less precise than a set of equations. The great 
196 
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flexibility of a flow chart permits incomplete or fuzzy statements. While 
this characteristic may be convenient in the early stages of model build- 
ing, at some point precision is required. For a flow chart to achieve 
precision, it must become the basis of a working computer program. 
Without a program, there is no way to tell whether or not the chart 
specifies the process exactly, for all possible contingencies. The program, 
rather than the flow chart, is the model. A digital-computer program is, 
by definition, a complete specification of a process, every bit as exact 
as a set of mathematical equations. In fact, the program, together with 
the logical design specifications of the computer, can always be expressed 
as a set of logical equations. If the equations were to be developed and 
written down, they would fill several volumes and would be about as 
valuable as the physical equations specifying the path of a falling feather. 
The program itself or a complete flow chart is the most concise way of 
stating these implicit equations. 

The notion that a digital-computer program could be a model of 
human behavior is a natural outgrowth of the field of engineering en- 
deavor that has come to be called artificial intelligence. Workers in this 
field are striving to develop more powerful machines, primarily by the 
automation of human functions. Automatic visual perceivers, automatic 
speech recognizers, and many kinds of learning and game-playing ma- 
chines have been built or programmed (see Minsky, 1961a, b). Most of 
these automatons make no pretense of achieving their results in the same 
way as their human counterparts, but a few have been made with the 
specific intent of simulating human behavior rather than just getting 
the appropriate outcome. Many workers feel that the best road to a 
really intelligent machine is the careful simulation of human mental 
processes. But any automaton, whether it is intended to simulate human 
behavior of just to do manlike things, is by definition a model of behav- 
ior. If a machine accomplishes the same result as a person, the machine 
is manifestly a model of human behavior, as Boring (1946) has pointed 
out so lucidly. The model may not mirror all aspects of the human 
behavior in question—it may not make the same mistakes that men 
make—but the model may serve as the starting point for a more inclu- 
sive psychological model. It would be surprising if work directed at 
producing a clever machine would yield acceptable psychology, but it 
would also be surprising if psychology could not profit from this work. 

Some psychologists have already taken the opportunity to begin 
fashioning models of human behavior in the form of computer programs. 
Information processing models of problem solving, rote verbal learning, 
concept formation, attitude change, and small-group behavior have been 
reported, and many more are being formulated, These psychological 
models should be distinguished carefully from the artificial-intelligence 
programs that are models only by accident, The latter should be judged 
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by the cleverness of the program’s behavior and should be taken as 
merely suggestive for psychology, whereas the serious: psychological 
models should be judged by the usual scientific criteria of aptness, 
goodness of fit, and parsimony. Applying these criteria to information 
processing models creates new problems, but that is the price of progress. 
An information processing model appears to be the best, and often the 
only, kind of model that can reasonably be used for complex behavior 
with many contingencies. 

The role of the computer in information processing models is some- 
times misunderstood. In artificial intelligence the computer is the focus 
of attention—the effort is to make a powerful machine. But the makers 
of information processing models use computers only as an instrument 
in testing the model. The computer plays the same role here that it 
plays in Monte Carlo analyses: the computer puts the model through its 
paces and records the results. In both situations, the job could, in 
principle, be done without a computer, though the cost would be pro- 
hibitive. Indeed, designers of information processing models often resort 
to “hand simulation” of the computer program, by which they mean 
manually executing the symbol manipulations of the model to see what 
happens, before they go to the trouble of writing the program. In fact, 
hand simulation could be used exclusively, if it were done by some dis- 
interested person following explicit rules. Doing the work on the com- 
puter has the advantage of ensuring objectivity and explicitness and of 
producing many more results much more cheaply, once the program 1s 
written and debugged. 

In this chapter we shall examine both information processing models 
and artificial-intelligence programs, to demonstrate the broad spectrum 
of possibilities in computer modeling. We shall try to distinguish between 
the programs that are models by design and those that are models by 
accident, but the distinction is not always clear-cut. The field is young 
and confused, and some scientists wear both hats. This is definitely a 


chapter of beginnings. 


11-1 Models of neural activity 


For centuries the brain has been an intriguing puzzle for scientists, 
Anatomists know that the brain contains billions of neurons arranged in 
various large clumps, such as medulla, hypothalamus, and cerebral cor- 
tex, and that within each gross structure there are f urther stratifications 
of neurons, but detailed knowledge of the wiring diagram is very sketchy. 
Physiologists know the functions of various areas of the brain but do not 
understand very much about how the functions are carried out. The 
intricate and complex network of neurons that make up the cerebral 
cortex is especially puzzling, How is this network organized to enable 





Computer models of psychological processes 199 


the organism to perceive, to learn, and to solve problems? The organiza- 
tion must be partly predetermined, at least for motor and sensory 
processes. But the faculties of perception and memory develop gradually, 
and the organism is continually learning; so it would seem that the 
network of neurons in the cortex must be continually reorganizing and 
developing new structures. How can this development occur? 

There are two approaches to this big problem. The classical approach 
is to continue physiological studies of neural activity. Technical ad- 
vances in constructing microelectrodes and in making electrical record- 
ings of neural activity have stimulated this physiological effort in recent 
years. Still, techniques for recording the individual activity of several 
cells simultaneously are only now being considered, and the physiological 
study of networks of neurons seems a long way off. A number of impa- 
tient investigators have taken a different approach. They have set up 
computer models of neuron networks and have examined the behavior 
of the models. These workers, many of whom are electrical engineers 
and logicians, have been dubbed ‘‘dry”’ physiologists, in contrast to their 
more orthodox colleagues, the ‘‘wet’’ physiologists. 

Models of neural activity antedate digital computers. Rashevsky 
(1938) and his group of mathematical biophysicists, including House- 
holder and Landahl (1945) and others, proposed models specified by 
differential equations for the energy exchanges and other temporal char- 
acteristics of neurons and synapses between neurons. These models are 
difficult to study mathematically, and the difficulty is compounded if 
more than one or two neurons are considered simultaneously. McCulloch 
and Pitts (1943) started considering networks of neurons at a time when 
digital computers were just appearing on the scientific scene. Further 
impetus to the study of neural models was given by Hebb’s book (1949), 
with its hypothesis about cell assemblies, formed in response to external 
stimulation and maintained autonomously by the continual firing of the 
neurons in the assembly. More recently von Neumann (1957) made some 
interesting observations about the similarities and differences of the 
computer and the brain. 

At an abstract level, neurons are simple all-or-none devices and are 
very similar to the binary elements in digital computers. Consequently, 
the mathematics of logical design that is applicable to computers also 
has some bearing on neural networks. The mathematical representation 
of neural networks has been studied by Kleene (1956), Burks (1960), 
Keller (1961), Uttley (1956), and others. The analysis is esoteric, and 
the problem is very difficult, It is much easier to study neural networks 
by building models. 

Some dry physiologists have constructed artificial neurons out of elec- 
tronic components and have strung these neurons together in networks 
(see Babcock, 1960; Harmon, 1961, 1962), Others have found it more 
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convenient to specify the nature of the behavior of individual neuronlike 
elements and to simulate a network of such elements with a digital- 
computer program. Changes can be made more simply by altering the 
program than by rewiring the network or changing components in the 
artificial neurons. 

The computer models of neural nets are all very similar to each other. 
The hypothetical nets are made up of neuronlike structures that we shall 
call elements, or cells. Each element receives impulses from several ele- 
ments and sends impulses to other elements. All connections are one-way 
streets; an element does not, 1n general, send its output to the same cells 
from which it receives inputs. It should be emphasized that a cell in 
such a model does not correspond with a binary-computer element. 
Rather, a cell is an entity in a computer program that is simulating the 
network. Each cell has several characteristics, and at any time there is a 
particular value associated with each characteristic. A cell is represented 
in the computer by a set of storage registers containing the current values 
of the cell’s characteristics. During the operation of the computer pro- 
gram, values are changed periodically, in accordance with the rules 
of the model. 

There are three kinds of cells: stimulus cells, response cells, and asso- 
ciation cells. Stimulus cells receive their inputs from the environment. 
They have no input connections from other elements in the network. 
The outputs of stimulus cells, however, are connected with other cells 
in the normal way. Response cells receive their inputs from other cells 
in the network, but their only output is to the external environment. 
These outputs are the responses of the system which are received by the 
experimenter. Between the stimulus cells and the response cells lies a 
complex network of association cells, with both input and output con- 
nections. The interconnections among the cells in a network are speci- 
fied at the start of a simulation and remain fixed, although the specifica- 
tion may be determined by random processes. 

The impulses received by an element are of various amplitudes, each 
represented by a digital number. The total input to the element is the 
sum of the input amplitudes. If this sum is greater than a parameter 
representing the threshold of the element, the element fires, sending an 
impulse to each of the elements to which it has output connections. The 
threshold of an element is not constant. Immediately after the element 
fires, its threshold rises to infinity, representing the absolute refractory 
phase of a neuron. Then the neuron enters the relative refractory phase, 
during which the threshold gradually decreases until it reaches its initial 
resting value. 


When an element fires, the amplitude of its output does not depend — 


on either the input amplitude or the threshold; the all-or-none law 
decrees that all firings are equivalent, Among wet physiologists this law 
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is currently in doubt. Evidence has been obtained of graded responses 
in parts of some neurons. But at least the all-or-none law is a first 
approximation to the truth, and neural models can safely assume that the 
amplitude of the output does not depend on the amplitude of the input. 
This does not mean that the output amplitude is constant. Rather, the 
amplitude of the output depends on a parameter associated with the 
output connection. Each connection has an associated amplitude—when 
an impulse is sent across the connection, it has the amplitude of that 
connection. Thus a given element produces outputs of many different 
amplitudes, one for each of its output connections. Some of the assigned 
amplitudes can be negative, simulating inhibitory connections. Since the 
decision as to whether a neuron fires is based on the algebraic sum of the 
received amplitudes, a negative amplitude will reduce the sum, thus 
lowering the chance that the neuron fires. 

Changes in the network, which may be called adaptation, or learning, 
can be effected by altering the amplitudes of the various connections. 
Raising the amplitudes is equivalent to positive reinforcement; lowering 
the amplitudes to negative reinforcement. Changes could also be effected 
by raising and lowering the thresholds of elements, but adjusting the 
amplitudes is usually felt to be more in keeping with the current under- 
standing of the human nervous system. The amplitude of a connection 
is viewed as a property of the synapse. Changes in the amplitude are 
assumed to be analogous to changes in the conductivity of the synaptic 
junction. In such a network, all memory resides in the synapse, and 
the effect of any one event is distributed among many synapses, in 
keeping with Hebb’s (1949) view of the brain. 

In a digital model of a neural network, time is quantized. The inputs 
and outputs of each element are calculated at every discrete step in 
time. Suppose that at time ¢) an element fires, producing outputs along 
its connections to other neurons. At the next time step t,, these outputs 
are assumed to have arrived at the cells to which they are connected, 
where they are combined with the outputs from other connected cells 
that may have fired at time f. The algebraic sum of the received ampli- 
tudes is the input to an element at time ¢. This cell fires at ¢, if the input 
is greater than the cell’s recorded threshold at that time. Thus in Fig. 
11-1, if cells A, B, and C all fire at time fo, the input to cell D at time t 
is +38 +1 —2 = +2. If the threshold of D at time t is +38, D will not 
fire. If, at to, cells A and B fire but C does not fire, the input to D at time 
t, is +4, which exceeds D’s momentary threshold of +3, and so D will 
fire at time t;. Other cells will fire at that time, and the process continues. 

Temporal summation is included in some models. The input to an 
clement at a given time has an effect over several time periods, rather 
than being discarded after the decision is made about firing the neuron. 
In successive time steps, the input amplitude received in the previous 
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time period is reduced by an amount proportional to the amplitude; then 
the new inputs are added to it. This rule is equivalent to an exponential 
decay of the excitatory input. 

Particular simulations of neural nets differ in the number of network 
elements, the number of interconnections from element to element, the 
regularity or randomness of such connections, and the specific rules for 
reinforcement, as well as in the assumed properties of the simulated 
neural elements. Some simulations are concerned only with spontaneous 
activity within the network; they have no specific stimulus or response 
cells. To start things going, the experimenter reaches in, as it were, and 
fires a group of cells. Then the network proceeds spontaneously. 

Very little systematic evidence has resulted from studies of simulated 


Association 
cell 





Fig. 11-1. Cell D receives input from stimulus cells A, B, and C, with weights 
+3, +1, and —2, respectively. 


neural networks. Rochester et al. (1956) showed that inhibitory as well 
as excitatory connections (negative and positive amplitudes) were 
needed for a certain randomly connected network to form cell assem- 
blies. Without the inhibitory connections, the network was inherently 
unstable. With almost any randomly connected network, it is apparently 
possible to produce reverberating activity that sustains itself for a short 
time and then dies out. Rochester et al. (1956), Clark and Farley ( 1955), 
and others have demonstrated this property. Networks with stimulus 
and response elements are able to learn some discriminations and not 
others. Clark and Farley (1955) demonstrated a form of generalization 
in a simple nerve net. 


_The perceptron. The most celebrated neural network is Rosen-_ 


blatt’s perceptron (1958, 1961), Widely heralded as the answer to the 
pattern-recognition problem, the perceptron is in fact a simulated net- 
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work of a simple type. The importance of the project lies in the many 
digital simulations and mathematical analyses done by Rosenblatt and 
his coworkers. For the first time, a consistent body of information is 
being amassed about the behavior of a simulated neural network. 

Rosenblatt has studied several types of perceptrons, each with differ- 
ent rules governing the interconnections of elements and the reinforce- 
ments of amplitudes, but most of the work has involved a simple network 
that Rosenblatt calls a three-layer series-coupled perceptron. All percep- 
trons include three kinds of neural elements: stimulus units having only 
output connections to other units, response units having only input 
connections from other units, and association units having both input 
and output connections. Each stimulus unit is connected to a fixed 
number of randomly chosen association units. The amplitudes of these 
connections are fixed a priori and do not change under reinforcement; 
some are positive (excitatory) connections; others are negative (inhib- 
itory) connections. The simple perceptron has only one output cell, 
which is connected to every association unit. The output cell either fires 
or does not fire: consequently the simple perceptron is response-limited 
to a 1-bit discrimination. In the simple perceptron there are no inter- 
connections among association units. All paths are of the form stimulus- 
association-response. 

The amplitudes of the connections between the association units and 
the response unit are altered by the reinforcement procedures. Three 
reinforcement procedures have been used: response-controlled reinforce- 
ment, in which the connections resulting in a particular response are 
reinforced so that that response will tend to recur when those connections 
are fired again; stimulus-controlled reinforcement, in which whatever 
response occurred to a particular stimulus will tend to recur when that 
stimulus is presented again; and error-correcting reinforcement, in which 
no reinforcement is given if the machine responds correctly, but negative 
reinforcement is presented if the machine responds incorrectly to a given 
stimulus. Reinforcement adds a constant increment (decrement for nega- 
tive reinforcement) to the amplitude of all connections that were active 
in the last time period; the amplitude of inactive connections is un- 
changed. In a slight variant of this scheme, the same increments are 
made, but then all amplitudes in the system are reduced by the average 
increment, so as to keep the sum of all amplitudes constant. The two 
systems give about the same result. 

Because the simple perceptron can make only a single discrimination, 
the problem posed for it is to discriminate between two classes of stimuli, 
a stimulus being defined as the firing of any particular subset of stimulus 
units. There are, of course, a great many ways to define two stimulus 
Classes from the set of all possible stimuli, and there is no a priori reason: 
why a perceptron should be capable of discriminating between all possi- 
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ble pairs of classes. However, Rosenblatt has proved that, if a dis- 
crimination is possible with a given network, then an error-correcting 
reinforcement procedure will eventually produce it, provided that all 
stimuli in the set of possible stimuli occur at least twice during the 
training procedure. Nothing is proved about the number of trials to 
solution. 

The general results of a large number of experiments and analyses 
of simple perceptrons show that they are able to make discriminations 
but that a large number of elements, and often a large number of trials, 
are required. For any given perceptron of the simple type, there will be 
some impossible discriminations; no assignment of amplitudes to inter- 
connections will enable that perceptron to make the discrimination. But 
if a perceptron is potentially capable of making a certain discrimination, 
then it can be taught to make that discrimination. Any sequence of 
training events is permissible, but certain sequences promote faster 
learning. Likewise, a perceptron can be trained no matter what its state 
at the start of training, but some states are better than others. However, 
a simple perceptron cannot generalize. The ability to generalize can be 
provided either by adding another layer of association units or by allow- 
ing interconnections among association units. These more complex per- 
ceptrons can also learn to recognize transformations that occur in a 
sequence of stimuli and can be made to exhibit selective attention to 
familiar objects. 

The stimulus cells can be arranged conceptually in a two-dimensional 
area, analogous to a retina. A two-dimensional form, e.g., the block letter 
X, can be set down on the area, stimulating the cells that it covers. The 
initial perceptron experiments showed that the perceptron could per- 
ceive the difference between block X’s and O's, no matter where in the 
“retinal” field the X’s or O’s occur. Nearly all possible X’s and O’s had to 
be presented during the learning trials, so that a large number of learning 
trials was required. Roberts (1960) has shown that by centering the 
stimulus on the “retina”? and by equating the sizes of all letters, thus 
effectively reducing the number of possible stimuli in the two classes, a 
perceptron can be made to learn very much more swiftly than in the 
more amorphous situation. Roberts also found it helpful to introduce 
some nonrandomness in the connections from stimulus units to associa- 
tion units and in the order in which stimuli were presented. These results 
typify the general principle that, the more regularity a network contains, 
the better it performs. 

It has been argued that the perceptron will eventually solve the 
pattern-recognition problem because all complex discriminations must 


ultimately be composed of a series of binary discriminations. A suffi- — 


ciently large collection of perceptrons should then be able to achieve any 
desired perception, This is much like saying that boards and nails solve 
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the problem of architecture because any building can be constructed by 
nailing enough boards together. Neural nets like the perceptron require 
hundreds of elements to make a 1-bit discrimination. Thousands of ele- 
ments would be required to recognize numerals and letters, and tens 
of thousands to recognize simple line drawings. Man’s perceptual ex- 
perience is so varied, and the catalog of recognizable objects is so vast, 
that a random mechanism would need an inordinate number of cells. 
Clearly some sort of organization must be imposed on the random nets 
to produce a machine capable of emulating human perception. 

Potential extensions of neural models. Most of the research on 
simulated neural networks has as its goal a machine that can, by adapta- 
tion and self-organization, learn to discriminate patterns and solve prob- 
lems (see Hawkins, 1961). Interest in the nervous systems of living 
organisms is secondary. Neurons are modeled because networks of 
neurons do seem to learn and to adapt to new stimuli. Other network 
elements could be used in place of neurons, and indeed this would seem 
to be a fruitful area for future research. 

Randomness is another property of networks that must eventually 
give way to more productive alternatives. Engineers and mathematicians 
have no need to resort to randomness to build signal detectors and in- 
tensity discriminators. Very fine signal detectors can be built that have 
fixed, carefully designed circuits, with each wire in its proper place. 
‘These circuits can be made adaptive by providing variable self-regulating 
criterion levels, with no random elements. Why, then, should random- 
hess seem a reasonable basis for more complicated, highly organized 
processes? 

Possibly the perceptron could serve as a part of a more elaborate 
model of cortical functions. It is as difficult to believe that the neural 
interconnections are completely predetermined as it is to believe that 
they are completely random. Perhaps perceptrons could be the elements 
in a hierarchy of processes that receive structured inputs from the 
environment and produce structured responses. The physiological evi- 
dence seems to demand a model with structured sensory inputs. Differ- 
ent kinds of information are transmitted on different neural pathways, 
and the nerve fibers appear to know what part of the sensory system 
they come from. (Lettvin et al., 1959; Maturana et al., 1960.) Further- 
more, some sort of hierarchy of stimulus-analyzing mechanisms seems 
needed for economy, as Sutherland (1959) has argued so forcibly. An 
excellent summary of recent physiological evidence about brain function- 
ing has been given by Pribram (1959). 

Attempts to provide more orderly organization in neural models might 
well focus on the simpler processes of signal detection and intensity 
discrimination, Seldom, if ever, have psychophysical experiments been 
conducted with simulated neural nets, Indeed, nets do not as a rule 





206 Behayioral-science applications 


make any provision for representing stimulus intensity, either by the 
number of elements activated or by differential rates of firing. Stimuli 
are seldom applied for more than a single discrete time period; so tem- 
poral effects of the stimulus cannot be watched. Including these effects 
in simulations would permit more direct comparison with psychological 
and physiological evidence. 

One possibility that the nerve-net builders seem to have overlooked 
in their enthusiasm for building giant human brains is that of modeling 
some very simple organism. Dethier (1955), for example, has amassed a 
great deal of evidence concerning the chemoreceptors of the ordinary 
housefly. The entire nervous system of this organism consists of only a 
few thousand cells, and if the visual system is eliminated, the remainder 
consists of only 400 neural cells, apart from the stimulus and response 
cells. Many of the particulars of the nervous system are well understood, 
and it would seem much more to the point to simulate such an organism 
in detail than to examine random models of man’s cortex. The random 
models are too divorced from the details of physiology to be very inter- 
esting to physiologists, and the behavior of the model is too simple to 
excite psychologists. Only by making serious efforts at modeling, in 
considerable detail, what is known of the physiology of the nervous 
system will any important progress be made in understanding the ac- 
tivity of living neural networks. 


11-2 Signal detection 


One of the simplest processes that is carried out by the human nervous 
system is typified by a psychophysical experiment in which the observer 
is asked to report the presence or absence of a particular signal, usually 
an auditory or visual stimulus. If the signal is very weak or is partially 
masked by noise, the observer can detect the presence of the signal only 
on some fraction of the trials in which the signal actually occurred. 
This characteristic behavior of the human senses is the subject of an 
information processing model that appears to have wide applicability. 
The model is sufficiently simple so that its analysis does not need a 
computer, but the model is a by-product of the development of auto- 
matic sensing devices and is thus in much the same position as other 
models based on clever machines. 

Many military systems have in the past made use of human operators 
to detect the presence of radar or sonar returns. The human observer is 
usually sufficiently sensitive to detect the weak signals of these systems, 
but his boredom, fatigue, and variability make him unreliable. These 


human failings have led systems engineers to design automatic devices” 


for detecting signals. Of course, simple devices for automatic detection 
have been known for a long time. A doorman can be replaced by an 
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electronic photocell; scales at airline terminals can signal when a passen- 
ger’s baggage weighs more than the 40-lb limit; some car radios have 
signal seekers that automatically tune the radio to strong signals in the 
broadcast spectrum. Automatic detectors for radar signals are not funda- 
mentally different from the simple devices; the signal is more compli- 
cated and its properties more difficult to measure, but the situation is 
otherwise the same. 

The theory of automatic signal detection is based on the statistical 
theory of testing hypotheses and on statistical decision theory. The 
theory as applied to radar signals was first developed by Lawson and 
Ullenbeck (1950). The theory was developed more fully by Van Meter 
and Middleton (1954) and Peterson, Birdsall, and Fox (1954). The aim 
of the theoretical studies is to specify the maximum attainable detect- 






Noise alone Signal plus noise 


Amplitude ——> t 


Decision 
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Fig. 11-2. Theoretical distributions for the amplitudes of noise alone and of signals 
in noise. The maximum-likelihood criterion is shown. An observation from the noise- 
only distribution that exceeds the criterion amplitude is a false report, while an 


observation from the signal-plus-noise distribution that falls below the criterion is 
. miss. 


ability of a signal for the given specified conditions. A machine attaining 
the maximum is an ideal detector. 

In its simplest form the theory supposes that the signal can be speci- 
lied by a single measurement, i.e., its amplitude. The signal occurs in a 
background of noise that may also be specified in terms of its amplitude. 
When a signal is not present, the observed amplitude is that of the noise 
alone, while if a signal is present, the observed amplitude is the combina- 
tion of noise and signal. By noise is meant any random variation in 
amplitude due to the background or environment of the potential signal. 
The theory supposes that, if the noise is presented a large number of 
times, its observed amplitude will have a statistical distribution that is 
assumed to be normal, Observations of signal-in-noise are also assumed 
to have a normal distribution of amplitudes, with a mean that differs 
from the noise distribution by a constant depending on the fixed ampli- 
tude of the given signal, This situation is depicted in Mig, 11-2, The 























208 Behavioral-science applications 


problem of signal detection is to decide whether the particular observed 
amplitude was drawn from the signal-plus-noise distribution or from the 
noise-alone distribution. The problem is analogous to the statistical 
problem of testing the significance of the difference between two means, 
and is also analogous to the method of paired comparisons in psycholog- 
ical scaling. Since the machine knows only the amplitude of the observa- 
tion, the best it can do is specify a cutoff amplitude above which an 
observation will be considered a signal and below which the observation 
will be considered to be noise alone. If the machine’s purpose is to be 
correct as often as possible, then a simple likelihood-ratio criterion will 
suffice. The obvious place to put the cutoff criterion is where the likeli- 
hood of a signal becomes larger than the likelihood of noise alone, as 
shown in Fig. 11-2. 

A noteworthy feature of this model of detection is its emphasis on all 
four possible outcomes: the signal may actually have been present and 
have been detected (a hit); the signal may have been present but failed 
to be detected (a miss); the detector may report the signal when in fact 
none was present (a false alarm); and, finally, the detector may correctly 
report that no signal occurred. In many practical applications the four 
outcomes may not have equal weight. In one situation, a miss might be 
more costly than a false alarm, whereas, in another situation, a false 
alarm could be disastrous. The detection model allows different values 
to be attached to the various outcomes. The criterion must then be ad- 
justed to maximize the overall value of the detector’s reports. To be 
more sure of hitting the target, the detector will have to give more false 
alarms, whereas, to avoid false alarms, an increased number of misses 
must be tolerated. 

The theory can be extended to include a variety of possible signals and 
many measurements on a given signal. Modern automatic detectors use 
many other properties of the signal than its amplitude in making their 
detections. However, the basic principles of the systems are the same as 
in the very simple example presented here. 

When the theory is applied to human observers, the observer is simply 
put in place of the machine. The best possible performance comes from 
an ideal observer, and actual observations are compared with this ideal. 
A presentation of a stimulus in a psychophysical experiment is equated 
with an observation in the theory. According to the model, the human 
observer receives the stimulus, compares its amplitude with his criterion, 
and makes his report accordingly. 

The most important contribution of detection theory to psychophysics 
is the treatment of false alarms. Traditionally, false alarms have been 
considered mistakes that are to be avoided. No attempt was made to 
measure the false-alarm rate systematically. Rather, catch trials were 
presented with the signal absent; if the observer reported a signal present 
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in more than a few of the catch trials, his data were discarded as being 
unreliable. Some experimenters have used the data but have adjusted 
the detection rate according to the number of false reports by applying 
the so-called “correction” for guessing. These experimenters assume that 
there is a clear distinction between a hit and a false report: either the 
observer heard the tone (saw the light), or he did not. If he heard it, he 
said so, but if he did not hear it, he sometimes guessed that it was there 
anyway. According to detection theory, the false alarms carry just as 
much information about the sensitivity of the observer as do the hits 
and misses. A trial in which only noise is presented is just as important 
as a trial in which the stimulus is presented. A false alarm is not evidence 
of an unreliable observer, nor does it indicate a ‘“guess.’’ According to 
detection theory the observer’s sensations are not dichotomous but are 
on an amplitude continuum. 

The main results of applying detection theory in psychophysics have 
been reviewed by Swets (1961) and by Swets, Tanner, and Birdsall 
(1961). The pioneering work of Tanner, Swets, D. Green, Marill, Egan, 
Clark, and many others has shown that the model applies very well to 
the human detection experiments. The most controversial result is that 
the human observer is much less sensitive than the ideal observer when 
detecting signals in a background of noise. Many reasons have been ad- 
vanced as possible explanations. The observer’s criterion may not re- 
main fixed over a large number of trials, introducing variability in the 
criterion point, noise may be added to the signal by the nervous system, 
or the observer may not be able to keep sharply tuned to a given input 
signal. But the most reasonable explanation would seem to be that the 
human observer is not designed to be ideal for detecting any one par- 
ticular signal in a noise background. Human beings are broadband de- 
vices, able to detect a very large variety of signals, and should not be 
expected to be completely optimum for detecting any particular chosen 
signal. It is true that in the absence of a noise background the human 
sense organs seem capable of detecting about the weakest signals that 
can possibly be produced, but perhaps the noise environment requires 
the use of different detecting processes. 

Not all automatic signal detectors are as simple as those we have sug- 
gested here. Some require fairly sophisticated analysis to detect the 
presence of a signal. For this analysis, computer techniques are invalua- 
ble. For example, in radar observations of the planet Venus made at 
the Lincoln Laboratory in 1958 (Price et al., 1959) a single radar return 
from Venus could not be reliably detected. Consequently, a large number 
of successive returns were recorded and added together by a process of 
integration in order to make the signal stand out from the noise back- 
ground, Perhaps similar signal processing occurs in human sensory sys- 
tems, As we gain more information about complex automatic detection 
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and about the human nervous system, more complex and. inclusive 
theories will become feasible. 


11-3 Pattern recognition 


Pattern recognition is a more complex process than detection. Detection 
is concerned with the presence or absence of one given signal, whereas 
pattern recognition is concerned with discriminating one out of a large 
number of specified signals. Pattern recognizers are designed to function 
in a context where the set of alternatives is known, such as the letters 
of the alphabet or the phonemes of American speech, and the machine’s 
job is to categorize each particular input as one of the known alter- 
natives. There is generally no “‘noise alone” category as in signal detec- 
tion; rather, noise serves to confuse the categorizations of the machine. 

The simplest automatic pattern recognizers perform no signal analysis. 
They merely compare the raw input with a stored ideal version of each 
alternative, and the closest match wins. For example, a machine for 
reading printed numerals compares the input with a stencil of each nu- 
meral; the most closely fitting stencil indicates the choice. The notion 
behind these machines can be traced back to Plato, who argued that 
every concept had a corresponding ideal. Unfortunately some of the 
platonic ideals are very similar to others. Devices for reading printed 
numbers from checks surmount this trouble by creating artificial differ- 
ences in size and shape between the numerals, to accentuate the differ- 
ences. But if there is no opportunity to manipulate the environment, the 
stencils may overlap too much. Moreover, stencils for matching raw 
inputs become almost useless for recognizing human products, such as 
speech or handwriting, because of the large individual differences. There 
is no ideal, hand-lettered A or ideal spoken aye that will correspond 
closely with the products of different people, or even the same person 
on different occasions. Some analysis of the input is required. 

The strategy that has worked best in analyzing the inputs has been 
to choose several attributes of the alternative patterns and to charac- 
terize the inputs as well as the alternative patterns in terms of these 
attributes. One attribute of hand-lettered characters might, for example, 
be number of straight lines. Each pattern has a particular value of each 
attribute; thus A has three straight lines. The combination of an attiri- 
bute and its value will be called a property, so that A has the property 
of three straight lines. In the best of all possible worlds, each pattern 
to be recognized would have a unique set of properties and the properties 
of the inputs could be matched against the stored properties of the 


patterns, in much the same way that the simpler pattern recognizers ” 


match stencils. 
In practice, there is no single list of properties that characterizes all 
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instances of a given pattern. Rather, the instances give rise to a dis- 
tribution of values for each attribute of the pattern. The pattern-recog- 
nition program for the computer either must be given a distribution 
a priori for every attribute of every pattern or must develop the dis- 
tributions as it experiences a large number of different instances of the 
pattern. To recognize an object, the machine must pick the likeliest 
alternative by determining, for each stored pattern, the likelihood that 
the object is an instance of that pattern. This is done by comparing the 
observed properties of the object with the stored distributions of values 
for that pattern, determining the likelihoods that each observed prop- 
erty is a member of the corresponding distribution, and combining the 
individual likelihoods in some way. 

One of the simplest pattern-recognition devices is MaupE (Morse 
AUtomatic DEtector), described by Gold (1959), which automatically 
recognizes hand-sent Morse code. Maupr discriminates between dots 
and dashes and also has the problem of discriminating among the three 
kinds of spaces: spaces between dots and dashes within a letter (symbol 
spaces), spaces between letters, and spaces between words. The durations 
of these three types of space are supposed to be in the ratios of 1:3:7, but 
human senders tend to slur these distinctions. The only attribute that 
Mavpr uses is duration, and its use of this attribute is very similar to 
the use of amplitude in the simple signal detectors described in the pre- 
ceding section. The distributions of durations are formed for each symbol 
and space, and criteria are selected for deciding among the alternatives. 
For dots and dashes, the distributions overlap very little, and the cri- 
terion is easily set, but there is much more overlap among the three 
distributions of the three spaces. Maupr adjusts its criteria as it goes 
along, according to local rules. For example, among six successive spaces, 
at least one must be a letter space, because no letter in Morse code has 
more than five symbols. 

Most pattern-recognition situations require the use of more than one 
attribute. A pattern is defined as a collection of properties. As such, a 
pattern is formally identical with a concept in most concept-formation 
experiments (e.g., Bruner et al., 1956). In these experiments each stim- 
ulus consists of several attributes, e.g., color, number, shape, size. The 
subject’s task is to learn which attributes are relevant to a particular 
concept and which values of these attributes constitute instances of the 
concept. Most pattern recognizers that use the property principle could 
be ealled probabilistic concept formers. They are presented with many 
instances of each pattern and keep a record of the properties of these 
instances. ‘The common properties are abstracted to form the concept. 

Selfridge (1959) has described a very colorful model for the property 
approach to pattern recognition, which he calls Pandemonium. The 
model is shown in Fig, 11-3; it consists of a large number of demons at 
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various levels in a hierarchy. A decision demon sits at the top level, 
examining the output of a set of cognitive demons each representing one 
of the alternative patterns and shouting with a loudness indicating the 
extent to which the input matches its pattern. The decision demon 
merely picks the loudest shouter. Each cognitive demon is connected to 
several computation demons that are examining the properties of the 
input pattern. Each computation demon examines the data to deter- 
mine whether or not that particular property exists. If it does, that 


Decision demon 








Cognitive 
demons 


Computation 
demons 


Data (input pattern) 


Fig. 11-3. Pandemonium. 


demon produces an output to all the cognitive demons to which it is 
connected. In general terms, as Selfridge puts it: 


The computational demons perform certain more or less complicated compu- 
tations on the data and pass the results of these up to the next level, the cog- 
nitive demons who weigh the evidence, as it were. Each cognitive demon 
computes a shriek, and from all the shrieks the highest level demon of all, 
the decision demon, merely selects the loudest. 


In Selfridge’s model the connections between the computational demons 
and the cognitive demons have weights associated with them, propor- 
tional to the likelihood that that property represents an instance of the 
particular pattern represented by the cognitive demon. The weights 
then represent either the a priori distribution of properties over patterns 
or the experience of the machine with a set of known patterns. 
The.Pandemonium model is very general. For example, by removing 
the computational demons and connecting the cognitive demons directly 


to the input data, we have a formalization of the very simple stencil- — 


matching pattern recognizers. Also, because very little is said about the 
kind of computations done by the computational demons, a large variety 
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of possibilities is encompassed. Moreover, the model can easily be com- 
plicated by adding additional layers of demons between the computa- 
tional and the cognitive demons. Certain properties may themselves be 
combinations of other more basic properties. There is no feedback in 
Pandemonium. All properties are examined in parallel and the results 
communicated to all the cognitive demons at once. This approach can 
be contrasted with a serial arrangement of decisions, in which the out- 
come of one demon determines which other demons get to see the input 
next. In analyzing handwritten letters, for example, a serial system 
might first determine whether or not any curved lines were present on a 
given input, initially dividing the possibilities into those letters composed 
entirely of straight lines and those containing at least one curved line. 
Different properties would be examined in the two cases further to re- 
duce the number of possibilities, until a final decision was reached. 
It seems likely that an adequate model for human pattern recognition 
will have to include both parallel and series processing with feedback 
at several levels, so that decisions initially made can be undone if further 
results clearly show that a mistake has been made. Also, any model of 
human recognition will have to consider the problem of context; thus a 
letter that in isolation might be either a or r will certainly be seen as a 
if it occurs as the fifth letter in Massachusetts. But whatever their short- 
comings, present automatic pattern recognizers focus attention on hu- 
man perception as a process and presage more sophisticated information 
processing models of human pattern recognition. 


Illustrative example: hand lettering. Doyle (1969) used Pandemonium 
as a model for his computer program for recognizing sloppy hand-lettered char- 
acters. The data were collected haphazardly by asking passersby to write their 
names and addresses, letter by letter, in a set of squares on a simple question- 
naire. Each figure was scanned by a photocell device and transformed into a 
series of dots on a 32 X 32 array. A sample is shown in Fig. 11-4a. Before ana- 
lyzing the properties of the character, the computer cleaned up the figure by 
filling in holes and deleting isolated dots, as shown in Fig. 11-4b and c. Following 
this processing, which was suggested by the pioneering work of Selfridge and 
Dinneen (1955), 30 attributes of each figure were examined. Some of the attri- 
butes were: the maximum number of intersections of a vertical straight line 
with the figure; the maximum number of intersections of a horizontal straight 
line with the figure; the number of cavities facing left, facing up, facing down, 
and facing right. Some attributes had only 2 or 3 possible values; others had 
as many as 10 or 20, 

Doyle selected 10 letters, A, E, 1, L, M, N, O, R, 8, T, for which many 
instances were available from the collected data, He split the data into two 
parts, using one part to develop the property distributions for the recognizer 
and the other part to test the results of the recognition process. Doyle presented 
one set of figures to the computer, together with labels specifying which letter 
cach square was supposed to contain, ‘The computer determined the values of 
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each of the 30 attributes and tallied these results in a set of distributions for 
that character. A small portion of the resulting set of distributions is depicted 
in Table 11-1. Thus property 2 occurred in 29 of the presented A’s, 34 of the 


Table 11-1 Portion of property distributions from Doyle’s pattern 


recognizer 
a 
Property A/E]I|LIMIN]O}]R]S]T Sum 


1 3] 24] 8 11 46 
2 29] 34} 1] 14] 1] 2] 28] 21} 26 | 24 180 
3 91.9 1) 12] 26} 6|10] 1] 8 82 
4 1 10} 1 2 2 16 
5 


1 1 


a eat 


presented E’s, 1 of the presented I’s, etc. In the recognition phase, the computer 
was presented with each character in the second set, without a label, and deter- 
mined the properties of the character. It then consulted the property distribu- 
tions to determine the likelihood of each letter. For example, Table 11-1 shows 
that a figure with property 2 has an a priori probability of ?%so of being an A 
and a figure with property 3 has an a priori probability of %o of being an A. 
A weighted average of these inverse probabilities was obtained for every possible 
pattern, and that pattern having the highest weighted average was chosen 
as the category of the presented character. 

The results of this scheme were encouraging; the computer recognized about 
87 per cent of the characters correctly. Human observers tested with the same 
characters in isolation recognized 97 per cent of the characters correctly (see 
Neisser and Weene, 1960); so there is room for improvement. Additional attri- 
butes or modification of existing attributes could be tried. Sherman (1959) has 
used the topological characteristics of letters as attributes, with considerable 
success. Kirsch et al. (1957) have suggested some processing of pictures that 
could provide new attributes. Harmon (1960) used different attributes for 
recognizing simple line drawings, and these attributes might be equally useful 
for letters and numbers. Doyle’s program can incorporate any attributes that 
will provide additional discriminations. The program is based on a catch-as- 
catch-can philosophy. This does not rule it out as a basis for a model of percep- 
tion. Very likely human perceivers are able to use whatever information they 
can get about a pattern, in a catch-as-catch-can fashion. 

Note that in Doyle’s program, and in the Pandemonium model generally, 
there is no use of scales or continua. The model is strictly nominal. Each attri- 
bute-value combination is taken as a separate property for which a distribution 
of instances must be developed. Clearly, if each attribute were a quantitative 
variable, then the concept of distance could be employed to reduce greatly the 
number of distributions required. It might be necessary to store only a single 
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distribution for each attribute. Moreover, if the distributions were unimodal 
and nearly symmetric, they could be characterized by a mean and standard 
deviation and a normal approximation could be used, rather than having to 
store the entire distribution. 





Fig. 11-4. Sample character from Doyle’s pattern-recognition program. (a) Origi- 
nal figure; (b) original figure after filling holes; (c) figure after deleting outlying 
dots from filled figure. The rules for filling and deleting may be stated with reference 
to a 3 X3 (ticktacktoe) grid centered on the given cell and lettered as follows: 


a b c 
d e f 
g h t 


If e is empty, it is filled when a and i, or b and fh, or ¢ and g, or d and f are both 
filled (or any combination of such pairs). If ¢ is filled, it is deleted unless one or 
more of the cells a, b, and d and one or more of the cells f, h, and 7 are filled or 
unless one or more of the cells b, c, and f and one or more of the cells d, g, and h 
are filled. 


Illustrative example: vowel-recognition program. A first step toward 
the goal of automatic speech recognition is represented by the vowel-recognition 
program of Forgie and Forgie (1959). Their program does not use the Pande- 
monium scheme; rather, it uses sequential rules for categorizing patterns. Like 
Doyle, the Forgies are willing to use any attributes that work. Some of the 
attributes are based on previous studies of speech recognition, others are based 
on empirical computer results, but all are ad hoc. A similar philosophy has been 
followed by Denes (1959) in his mechanical speech recognizer. 

The Forgies used a vocabulary of isolated words of the form b-vowel-t. They 
chose 10 vowels for study, but because the vowel in book did not occur naturally 
in English in the b-t context, both book and put were included to provide each 
end of the context separately. The computer was programmed to recognize 
the vowels in the following set: put and book, bit, bet, bot (as in bottle), bat, but, 
beet, boot, bought, and bert. 

The input to the computer was a tape recording of each of several persons 
speaking each of the English words. The tape recorder was connected to a bank 
of 35 bandpass filters that covered the frequency spectrum of speech from 115 
to 10,000 eycles per second, The filters performed a short-term frequency 
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analysis of the continuous speech wave from the tape recorder. Each filter put 
out a voltage proportional to the amount of speech energy in its frequency band. 
A rotary switch scanned the filter bank 180 times per second, connecting each 
filter in turn to an analog-to-digital converter; so every go second the computer 
had a look at the speech signal; i.e., it received a set of 35 numbers representing 
the amplitude of the signal in the 35 frequency bands. Figure 11-5 shows a plot 
of one look taken near the middle of the word bet for a male speaker. The program 
attempted to recognize the vowel represented by every look and then combined 
these recognitions into a single choice for the spoken word. 

The overall scheme of the program is shown in Fig. 11-6. The first step was 
to locate the first two formants of the spectrum. The formants are peaks in the 
speech spectra of vowels. In Fig. 11-5, the first formant F'1 is at about 650 to 
700 cycles, and the second formant F2 is at about 1,700 cycles per second. The 
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Fig. 11-5. <A typical look. This spectral output of the bandpass filters is from 
within the vowel portion of the word bet spoken by a male speaker. 


low-frequency peak at 215 cycles indicates the fundamental pitch of the voice, 
rather than a formant, and the higher peaks, if indeed they are more than ran- 
dom fluctuations, represent the third and higher formants. 

The computer program located F1 roughly in one of four frequency intervals, 
If F1 was in the lowest interval, between 200 and 300 cycles, then the word was 
assumed to be boot if 2 was relatively low and beat if F2 was relatively high, 
For the other three intervals of F1, /2 was categorized into one of three inter- 
vals giving a set of nine categories. Within each of the nine /'1-F2 classes, tests 
were devised to discriminate among the remaining confusions. For example, 
Fig. 11-7 shows illustrative spectra of vowels from the words bat and bet, both 
with an [1 in the interval from 645 to 890 and an F2 in the interval from 1,700 
to 2,400 cycles per second. The most noticeable difference is the valley between 
the two formants, which is lower for bet than for bat. If the amplitudes in the 
region from 990 to 1,700 are summed and compared with the amplitudes in the 
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Fig. 11-6. Structure of computer program that classifies each look as belonging to one of the 10 possible vowels. 
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region from 645 to 890, the ratio of these two summed amplitudes will be larger 
for bat than for bet. The computer formed this ratio and determined whether 
it was above or below some criterion value. All the tests at this level of the pro- 
gram were ratios. If more than two vowels had to be discriminated, then another 
ratio was needed for each extra vowel. An additional complication arose because 
the performance of the system seemed to depend on the fundamental frequency 
or pitch of the voice. Thus a separate program was devised to determine the 
pitch, and the criterion values for the ratio tests were adjusted according to 
the pitch. The Forgies originally set the criteria for their ratio tests on the basis 
of manual data analysis, but more recently they have programmed the computer 
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Fig. 11-7. Illustrative spectra of the vowels falling in the F'l region 645 to 890 
cycles per second and the F2 region 1,700 to 2,400 cycles per second. 


to keep distributions of ratio outcomes, so that the criteria could be set auto- 
matically on the basis of the machine’s experiences. 

The complete computer program consisted first of an overall amplitude de- 
tector to determine when -the vowel portion of the word began and ended 
(vowels are much louder than consonants). During the vowel portion of the 
word, each look was examined by the decision tree (Fig. 11-6), and one of the 
10 vowels was selected as the recognized vowel. After all the looks from the 
vowel portion were processed, the most frequent choice of the recognition system 
was taken as the final choice of the system; i.e., the majority ruled. Note that 
the program used a serial decision process for each look but a parallel process 
for a vowel, in the sense that the decision for one look does not influence the 
decision for any subsequent look. In a later version of the program, the majority 
rule was modified to take account of the duration of the vowel, Some vowels 
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are generally shorter than others so that duration alone would rule out some 
choices. A range of permissible durations was established empirically for each 
of the 10 vowels. The recognition program would not accept the selection of 
the majority rule unless the observed duration was within the range of permissi- 
ble durations for the selected vowels. If the first choice was rejected, then the 
other vowels were considered in the order of the number of looks selecting them. 
The first vowel for which the range of acceptable durations included the ob- 
served duration was taken as the selection of the system. 

The program without the duration test was used to recognize the 10 words 
spoken by 11 male and 10 female speakers. The program correctly classified 
88 per cent of the vowels. By modifying the majority rule according to the dura- 
tion of the vowels, the program’s behavior was improved to 93 per cent correct. 
Human listeners correctly classified 97 per cent of the spoken words. 

Recently Smith and Klem (1961) have used the Forgies’ data in a multiple- 
discriminant analysis. They considered each look to be a 35-variable observation 
and calculated the optimum linear and quadratic functions of the observations 
for discriminating among the vowels. This objective statistical procedure, with 
the quadratic functions, yielded 94 per cent accuracy, which compares favorably 
with the Forgies’ results. 


11-4 Problem-solving and game playing 


“Can a machine think?” “Are machines intelligent?”” Such meaningless 
questions have an immense fascination for laymen and experts alike. 
People feel threatened by the increasing power of machines, and their 
anxiety is gleefully enhanced by newspapers and science fiction. Cock- 
tail philosophers are willing to allay these anxieties by pointing out that 
machines can do only what they are instructed to do and cannot exhibit 
the creativity and novelty that are the nub of thinking and intelligence. 
Many psychologists would hold the view that thinking, like other human 
behavior, follows natural laws that will some day be understood and that, 
when the laws are understood, computers can be programmed to follow 
them, and thus to think. 

The difficulty rests with the concepts of thinking and intelligence. For 
many years psychologists have recognized that intelligence can usefully 
be regarded as a combination of several different factors or abilities. 
Thinking can also be. understood in terms of its components, such as 
symbol manipulation, problem-solving, planning, etc. When the compo- 
nents of intelligence and thinking are examined, it turns out that 
machines have been programmed to exhibit many of these features. 
Machines have been programmed to play games, from ticktacktoe to 
chess, and to solve many types of problems, including logic, plane geom- 
etry, formal integration, and trigonometry. Games can be played and 
theorems proved in an exceedingly stupid way, given enough computa- 
tional capacity, but most of the programa that we shall discuss behave 
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in fairly intelligent ways. Some are even intended as models of human 
cognitive processes. 

Problem-solving seems a more difficult task for computers than signal 
detection or pattern recognition. Of course, all computers can solve 
problems when given very explicit instructions. The question is whether 
or not they can solve problems without such direction. One scheme that 
people keep pondering is random behavior. Friedberg (1958) has pro- 
vided a convincing demonstration that a computer cannot solve prob- 
lems effectively when behaving at random. He gave a machine the task 
of programming another, very simple computer by choosing instructions 
at random. The simple computer was to produce a single output by 
operating on a set of two or three inputs, a task not very different from 
adding 1 and 1 to get 2. After 10,000 trials the machine still had not 
written the proper program. Attempts to modify the random-selection 
procedure, while still leaving it essentially random, were mostly unsuc- 
cessful, although the machine occasionally managed to solve the prob- 
lem by a great amount of blind trial and error. It is valuable to know 
that this problem-solving strategy is a poor one. 

Logic Theorist. Most of the work in computer models of cognitive 
processes stems from the pioneering effort of Newell, Shaw, and Simon 
(1957). As their first venture, they programmed a computer to prove 
theorems in elementary symbolic logic. Their program, which they called 
the Logic Theorist, was able to prove most of the theorems in chap. 2 
of Whitehead and Russell’s great treatise “Principia Mathematica” 
(1935). Starting with the axioms of the logic system, the machine first 
proved the simple theorems and then used these results to prove the 
more complicated theorems. 

A theorem to be proved consists of a statement in elementary sym- 
bolic logic. To prove the theorem, one must show that the statement is 
true, which is done by showing that it can be derived from the axioms 
of the logic system or from other proved theorems. The starting point of 
a proof is a true statement, either an axiom or a theorem, which is 
transformed into other true statements in accordance with the axioms 
and theorems until the transformed statement is the theorem that was 
to be proved. The problem for the computer is to choose an appropriate 
starting place and a sequence of transformations that yields the given 
statement. 

A straightforward way for the computer to find a proof would be to 
construct all possible proofs that start from an axiom or proved theorem, 
applying all possible transformations until it finally comes upon a proof 
that ends with the stated conclusion. The obvious difficulty with this 
scheme -is its length. Newell, Shaw, and Simon estimate that for an 
average theorem in elementary symbolic logic a computer would have to 
generate about 50 million possible proofs before finding the desired proof, 
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They call this exhaustive search process the British Museum algorithm, 
since they feel it is on a par with the celebrated procedure for writing 
all the books in the British Museum by equipping a corps of monkeys 
with typewriters and waiting patiently until their output by chance 
includes the necessary literature. 

Obviously an intelligent problem-solver would try only those trans- 
formations which had some chance of success. The machine would have 
to analyze the problem, discover certain of its relevant characteristics, 
and choose related transformations. The Logic Theorist uses several 
principles or devices, which its authors call heuristics, that contribute 
to the reduction in the average search to solution. First, it works from 
back to front, attempting to transform the given statement into one of 
the axioms or previously proved theorems. When this attempt succeeds, 
the inverse sequence of transformations constitutes the proof. The basic 
transformation of the program is substitution. If a simple substitution 
or series of substitutions will produce a proof, the machine can usually 
find the proof directly. It uses a heuristic of similarity to determine 
likely target theorems into which it will attempt to transform the given 
statement, by substitution. If substitution fails, it uses three other 
methods, known as detachment, forward chaining, and backward chaining, 
which serve the subsidiary purpose of generating subproblems such that, 
if the machine could prove them, it could prove the given theorem. 
Detachment says, ‘If A is to be proved and it is known that A is true if 
B is true, then the subproblem is to prove the truth of B.’”’ Forward 
chaining says, “If it is to be proved that A implies B, and if it is known 
that A implies C, then the subproblem is to prove that C implies B.” 
Backward chaining says, ‘If it is to be proved that A implies B, and if 
it is known that C implies B, then the subproblem is to prove that A 
implies C.”’ 

The heuristics are general principles and are not guaranteed to work 
in all cases. The similarity heuristics may sometimes rule out some of 
the possible transformations that would have led to proofs. Of course 
there are always many paths to a proof, and the heuristics will seldom 
rule out all possible paths. Nevertheless, on rare occasions the slavish 
following of the heuristic principles will result in the machine’s failure. 
One pays this price for a very significant reduction in the search time 
for the proved theorems. The main characteristic of problem-solvers is 
the ramification of the search for useful transformations. A given state- 
ment may generate several subproblems, each of which in turn generates 
subproblems, until one subproblem can be proved by substitution. The 
search tree might appear like lig. 11-8. The main force of the heuristics 
is to keep the number of branches within bounds, 

Wang (1960) has eriticized the Logie Theorist on the grounds that a 
much more elegant algorithm exists for finding proofs in logic, so that 
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the heuristic method, while much faster than the British Museum algo- 
rithm, is much slower than Wang’s algorithm. However, Wang’s methods 
are specific to a particular set of theorems, whereas the Logic Theorist 
can in principle be extended to a more general class of problems. More- 
over, by introducing heuristics, Newell, Shaw, and Simon laid the 
groundwork for modeling human problem-solving. Wang’s objective is 
quite different; he is pursuing mechanical mathematics, and in this 
context the Logic Theorist may seem quite weak. Nevertheless, we are 
inclined to agree with Minsky (1961a): 


. . . It seems clear that a program to solve real mathematical problems will 
have to combine the mathematical sophistication of Wang with the heuristic 
sophistication of Newell, Shaw, and Simon. 


Theoret to be proved 


Subproblems 


Sub-subproblems 


True theorem 
Fig. 11-8. A search tree generated by the Logic Theorist. 


General problem-solver. The basic structure of the Logic Theorist 
is not specific to logic. By abstracting the general features, Newell, 
Shaw, and Simon (1959) constructed a program capable of solving a 
variety of problems, including problems in logic. This program is called 
General Problem-Solver, or just GPS, and is coded in Ipt-v (see Chap. 5). 
GPS works forward from a stated premise to a stated conclusion, using 
the transformation rules of the system in which the problem is phrased, 
e.g., symbolic logic. GPS contains methods for generating subproblems 
and for keeping track of the search tree of subproblems as it goes along, 
It has a general framework for noticing differences between the premise 
and the conclusion and for applying transformations to the premise, but 
the specific differences to be noted and the specific transformations to 
apply are a part of the context of the problem. The context, which 
Newell, Shaw, and Simon call the task environment, must be furnished 
to GPS as a part of the problem. 

GPS is organized in terms of goals and methods for achieving those 
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goals. As depicted in Fig. 11-9, there are three major goals, each with an 
associated method. The first goal is to transform the premise into the 
conclusion or, more generally, to transform object A into object B. 
GPS’s method for doing this is to compare A with B. If they are identical, 


Goal I: Transform Object A into Object B 


Compare A and B; 
Find most important difference D 


D found 


Subgoal: 
Reduce difference D 


not achieved 
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Goal Ill: Apply Operator Q to Object A 








Compare A with conditions of Q; 
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Succeed 


Apply @ toA 
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D' found 


Subgoal: 
Reduce difference D’ 


not achieved 















achieved: 
A" produced 


achieved 


A’ produced Suerees 


not achieved 
pat Fail 


Fig. 11-9. Major goals and methods in GPS. 


the problem is solved or, in GPS’s terms, the method succeeds. If they 
differ, the most important difference is selected and a subgoal is estab- 
lished of reducing this difference. If this subgoal is achieved, A will have 
been transformed into some object, say, A’, that is more similar to B 
and the problem becomes one of transforming A’ into B, which is set 
up as a subgoal, If either subgoal is not achieved, the method fails. 
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Objects are transformed in GPS by means of operators. To reduce a 
difference between two objects (goal II), GPS searches for an operator 
Q that is relevant to reducing the particular kind of difference noted 
between the two objects. For example, if A contains more variables 
than B, an operator that reduces the number of variables is a likely 
candidate and GPS sets up the goal of applying that operator to object 
A. If the goal is achieved, A’ is produced. Otherwise, the method fails. 

The method for achieving goal III, the application of operator Q to 
object A, is similar to that of goal I. To attain this goal, a check is made 
to see whether or not Q can be applied directly. If so, it is applied, yield- 
ing A’. If not, then some differences have been discovered between the 
form of A and the required form of Q’s operand. In this case, GPS selects 
the most important difference, establishes the goal of reducing the differ- 
ence, and then tries to apply the operator to the resulting object. If that 
succeeds, a new object, say, A’, will have been produced. Otherwise, 
the method fails. 

As a simple example, consider the well-known riddle of the missionaries 
and the cannibals (Simon and Newell, 1961). Three missionaries and 
three cannibals must cross a wide river, using a boat that will hold no 
more than two persons. All six know how to row the boat. The problem 
is to get the entire party ferried across the river without ever allowing 
more cannibals than missionaries on either side, since those missionaries 
would be eaten by the cannibals. To put GPS in this context, specifica- 
tions are needed for the kinds of differences that may occur between 
objects and the operators that may be used to transform objects. An 
object is a list of the people on each side of the river and the location 
of the boat. The initial object, or premise, is symbolized on the first 
line of Fig. 11-10: three missionaries, three cannibals, the boat on the 
left side, and nothing on the right side. The program can notice some 
gross differences, such as six men missing from the right side, and some 
specific differences, such as one less cannibal on the right or one less 
cannibal on the left. With each kind of difference a list must be supplied 
of the operators relevant to reducing this difference. Thus, moving two 
cannibals and the boat from left to right is one of the operators relevant 
to reducing the difference of six men missing from the right. The list of 
operators relevant to a difference must be put in priority order, to indi- 
cate the most likely operators. The differences themselves must be put 
in order of difficulty so that GPS can try to reduce the most difficult 
differences first. This heuristic helps GPS to restrict the expansion of the 
tree of subgoals generated during the course of the problem. 

GPS is a recursive program. Figure 11-9 shows that some goals are 
subgoals of others and some are subgoals of themselves. As the program 
proceeds, a hierarchy of goals within goals is built up. Figure 11-10 
shows part of this structure for the missionaries and cannibals riddle, 
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The first goal is to move all the men and the boat from the left bank to 
the right. In comparing the desired object with the premise, GPS scans 
the list of possible differences (not shown) and discovers first that six 
men are missing from the right. Thus it sets up the goal of reducing that 
difference. From the list of operators relevant to reducing that differ- 
ence (also not shown), GPS selects the operator for moving two cannibals 
and a boat from left to right and sets up the goal of applying that 


Left: MMMCCCB ; Left: — 
Traleeten ae — } ~ eae ancdie} 
Difference: Six men missing from right 


Apply (Left CCB ——~> Right CCB) 


Left: MMMC 
Produce fete CCB i 
Left: MMMC| . Left: — 
Heepsioen fae CCB } ae feos vances 
Difference: Four men missing from right 


Apply (Left CCB ——~> Right CCB) 
Difference: No boat on left. 


Apply (Right CB ——> Left CB) 


. Left: MMMCCB 
Produce: eee Cc } 


Left: [MMM 
Produce Right: ae 


Left: MMM] | Left: —— 
Tester ie a me a ances 


etc. 


Fig. 11-10. Part of a goal structure in GPS. The beginning steps in the solution 
of the missionaries and cannibals riddle are shown. The structure of the hierarchy 
of goals is indicated by indenting. M = missionary; C = cannibal; B = boat. 


operator. Nothing stands in the way; so the operator is applied, leaving 
three missionaries and a cannibal on the left and two cannibals and the 
boat on the right. A subgoal is now set up of transforming this object 
into the desired object. Now GPS finds that four men are missing from 
the right and tries to apply the operator of moving two cannibals and 
the boat from left to right, It finds that it cannot apply the operator, 
because there is no boat on the left, There is also only one cannibal 
on the left, but GPS does not notice this difference, because the lack of 
the boat happens to have priority, Luckily, the first method for moving 
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the boat is to send a cannibal across with it, after which two cannibals 
and the boat can go from left to right, as hoped. This leaves three 
missionaries on the left and three cannibals on the right with the boat. 
The next subgoal is to transform this state of affairs into the final desired 
object, and the program proceeds. 

GPS is controlled by an executive routine that keeps track of all the 
subgoals and reviews each new subgoal as it is generated. If the new 
subgoal is identical with an earlier subgoal, its present occurrence may 
be rejected. A transformation may be rejected as inadmissible; it might, 
for example, result in the cannibals outnumbering the missionaries on 
one side of the river. Also, a difference generated at one level in the re- 
cursion may be more difficult, in some defined sense, than the difference 
at the higher level in the hierarchy. An increase in complication looks 
like regression rather than progress to GPS; so the executive terminates 
this line of attack. Records are kept of each goal’s execution so that the 
program can try the goal several times, getting a different result each 
time. Thus, if the most important difference cannot be reduced, another 
difference can be selected. Normally GPS follows through a potential 
sequence, step by step, as far as it can without trying any other possi- 
bilities, until it bumps its nose. At that point it backs up one step and 
tries other paths. Eventually all these paths fail, or one succeeds. If 
they all fail, it backs up another step and tries further. 

Newell, Shaw, and Simon (1960) have suggested that GPS can be 
made to learn how to solve problems by generating for itself the lists 
of operators that are relevant to each difference and by establishing the 
priorities on these lists by trial and error. Many other sophistications 
are possible, and indeed GPS is a bit more sophisticated than this short 
discussion might imply. GPS has mainly been used in solving logic prob- 
lems, but it has also been used for trigonometry and for the missionaries 
and cannibals riddle; so a measure of generality has been demonstrated. 

Other problem-solvers. A few other problem-solving programs 
have been written. Gelernter et al. (1960) used their ForTRAN List 
Processing Language (see Chap. 5) to write a program for proving theo- 
rems in plane geometry. Like the Logic Theorist, the geometry machine 
worked backward, finding subproblems that, if solved, would enable the 
machine to prove the stated theorem. The main heuristic in the program 
involved the construction (symbolically) of a diagram of the given con- 
ditions. Relationships that appeared to be true in the diagram were 
considered as possible subproblems to be proved in order to prove the 
stated theorem. The geometry machine used many of the same methods 
as the high-school student and was able to get an excellent grade on @ 
final examination in high-school plane geometry. 

Slagle (1961) wrote a program in Lisp (see Chap. 5) that he calls 
SAINT, an acronym for Symbolic Automatic INTegrator. Sainv does for 
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freshman calculus what the geometry machine did for high-school plane 
geometry. It performs symbolic integration, including indefinite inte- 
gration and certain definite and multiple integrals that are trivial exten- 
sions of indefinite integration. SAINT uses the same methods that fresh- 
men are taught and grades very high on problems taken from the 
freshman course in mathematics at MIT. 

In a quite different context, Tonge (1961) wrote an IpL-v program to 
balance assembly lines. The problem arises in industries where the 
product is assembled on a continuous conveyor line. The elementary 
tasks making up the assembly operation must be assigned to work 
stations along the line, so as to obtain efficient operation of the entire 
line. Given the length of time required to perform each elementary task 
and the partial ordering that exists among tasks (some tasks cannot be 
done until others have been done), the program seeks to group the tasks 
into work stations so that a stated production rate is met, the number 
of work stations is reasonably small, and the amount of work assigned 
to the various stations is as even as possible. 

Simulation of human problem-solving. Psychological models 
deal with the products of organisms: a left turn in a maze, a response in 
a Skinner box, a remembered nonsense syllable. Human problem-solving 
cannot be studied effectively if the final solution to the problem is the 
only product. We are really interested in the processes by which the 
subject solved the problem. One way to get access to these processes is 
to augment the subject’s products by asking him to think aloud as he 
works on the problem. The verbal report is recorded; this protocol serves 
as the product to be modeled. 

Automatic problem-solvers can also be set to produce protocols. GPS, 
for example, can be programmed to print out every attempt to apply 
an operator and to print the result of this application together with the 
“reason” for choosing the operator. The problem for the modeler is to 
adjust the parameters of the program so that the machine will produce 
a protocol closely matching a given human protocol. In GPS adjustments 
can be made in the order in which differences are noted, in the transfor- 
mations that are considered relevant to each difference, and in the order 
in which the relevant transformations are tried. 

Newell, Shaw, and Simon (1958b) have proposed GPS as a model of 
human problem-solving and have used GPS to model the protocols of 
novices solving problems in logic (Newell and Simon, 1961). The prob- 
lems for the experimental subjects were phrased in terms of arbitrary 
symbolic codes, so that the subjects did not have to learn symbolic logic. 
Thus the expression (2 > ~ P) is read as ‘7? horseshoe tilde P”’ rather 
than “2 implies not P.”’ In this framework, a problem involved recoding 
one expression from another, by using certain coding rules, The protocol 
of an individual is used to determine the particular heuristics that he 
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seems to be exhibiting, i.e., the transformations that he seems to prefer, 
and the order in which he notices differences between the premise and 
the conclusion. In several cases the machine simulates very closely the 
detailed choices made by subjects. While Newell, Shaw, and Simon have 
very little systematic evidence about the fit of their model of problem- 
solving, their work is highly suggestive. Laughery (1961) and Johnson 
(1961), using more well-structured problems, have also obtained very 
encouraging results. 

There are serious problems involved in evaluating the extent to which 
an information processing model fits the data from a protocol. In many 
cases it is difficult to specify precisely what constitutes the parameters 
of the model. Which parts of the computer program shall be regarded as 
variable, and which shall be regarded as fixed? How can an objective 
procedure be defined for examining the protocol to determine these 
parameters, many of which are ordinal relationships? In any particular 
case, ad hoc evaluative procedures can always be set up, and as more 
persons become involved in devising computer models, the more general 
problems in evaluating the models will eventually be solved. 

The Turing test has often been mentioned as a strong test of simula- 
tion. A set of machine protocols and a set of human protocols are ob- 
tained and given to human judges. If the judges cannot tell which 
product is from the machine and which from the human, the simulation 
is deemed successful. But there are so many extraneous factors involved 
in such a judgment and the criterion is so unspecific that most psychol- 
ogists would prefer more direct methods of evaluation. 

Game playing. Computer programs that can play games such as 
checkers, chess, and the like, are very similar to theorem-proving pro- 
grams. The premise consists of the initial conditions of the game, e.g., 
the board and the initial position of the pieces in chess or checkers; the 
objective is simply to win the game. In place of the transformations we 
have the rules for playing the game, e.g., the legal moves that each piece 
may make in chess. The problem of searching for the right combination 
of moves is equivalent to searching for the right sequence of transforma- 
tions in proving theorems. Of course, most games cannot be played in 
reverse, from the final position back toward the initial position, as some 
of the theorem provers are able to do, since there are many possible final 
positions all of which result in the machine’s having won the game. It is 
also possible to consider each individual play in the game, e.g., each 
move in chess, to be a separate problem, the premise being the conditions 
of the game at the time of the play, and the objective being the best 


play at that time. A complete game, then, is composed of many problems, — 


In a simple game like ticktacktoe all possible sequences of moves can 
easily be considered. However, in a game of any complexity, the number 
of alternative sequences becomes tremendous, and some means of limite 
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ing the search must be found. Most game-playing programs do not follow 
through a complete sequence of potential moves in order to choose a 
single move. Instead, the machines look ahead only a certain number 
of moves, and that number is termed the depth of the analysis. Hach 
potential move constitutes one unit of depth. A machine that considers 
only the opponent’s probable countermove in choosing its own move is 
analyzing to a depth of 1, while a machine that considers its opponent’s 
move, then its own countermove, is analyzing to a depth of 2, and so on, 
Most game-playing machines have an arbitrary limit on the depth of 
analysis. 

Limiting the depth of analysis is not sufficient for most games, be- 
cause of the many possible moves at each level. Additional heuristics 
are needed to permit the machine to consider only the more likely moves. 
If a machine were to consider as many as 10 moves at each level, it 
would be examining 100 possibilities at the second level, 1,000 at the 
next, and 10,000 at the next. The value of limiting the number of moves 
considered at each level is obvious. 

Chess is a favorite game for machines to play, because it poses such a 
challenge. In the eighteenth century a chess machine was exhibited that 
played very well; but the machine’s main component turned out to be a 
clever midget. A legitimate chess machine of considerable power ap- 
peared two centuries later, in the form of an IBM 704 computer 
programmed by Bernstein and his colleagues (Bernstein et al., 1958; 
Bernstein and Roberts, 1958). The program looked ahead at a depth 
of 4 and considered the seven likeliest moves at each level. Several factors 
contributed to the likelihood that a piece would be moved: whether 
or not it was under attack, whether or not it could capture an opponent’s 
piece, whether it could move to a “better” position on the board, ete. 
‘The same kinds of considerations were involved in evaluating a par- 
ticular move; e.g., was the position improved? 

The most sophisticated chess-playing machine to date has been that 
of Newell, Shaw, and Simon (1958a). This program considers a variable 
number of possible moves, depending on the machine’s evaluation of 
their worth, and looks ahead to a variable depth depending on the 
fluidity of the situation as it progresses. Specifically the machine con- 
siders move and countermove until it reaches a “static position,’ one in 
which no piece may take another. The computer evaluates potential 
moves in terms of their utility in reaching intermediate goals, such as 
king safety, center control, and material development, rather than in 
terms of their contribution to the main goal of winning the game. Most 
of the decisions that go into a game of chess are too remote from the final 
outcome to be evaluated by that outcome, Mor the same reason, a chess 
machine cannot be programmed to improve its play if “win,” “lose,” or 
“draw”? is the only feedback, Learning seems possible only when heuris« 
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tics can be reinforced for leading to the achievement of some inter- 
mediate subgoal. 

Samuel (1960) programmed a computer to play checkers and to 
improve its play by a prodigious feat of memory. The program evaluated 
a checker position by computing a weighted average of the values of its 
attributes, such as mobility, advancement, and back-row control. The 
computer analyzed moves to a variable but limited depth and selected 
the move that led to the position with the maximum attainable value, 
assuming that the opponent always made his best move. The computer 
recorded each new position that it encountered, together with the value 
of the selected move, which was then treated as the value of that 
position. If the machine encountered the same checker position in 
analyzing another move, it automatically remembered the value of the 
position, so that the analysis was deeper than it would otherwise have 
been. As experience kept accumulating, the machine was, in effect, 
analyzing deeper and deeper, until it became an expert player. Remem- 
bering every event in the past may seem a brute-force kind of learning, 
but there is no doubt that the machine’s behavior improved. To provide 
the program with the ability to generalize from its experience, Samuel 
added a scheme for adjusting the relative weights of the attributes in the 
position evaluation. The effect of learning is summarized in the relative 
worths, so that positions that the machine has never encountered can be 
evaluated in the light of experience. 


11-5 Other psychological processes 


Information processing models of behavioral and social processes are 
only beginning to appear in the literature. Some of these will be discussed 
here, as an indication of what can be done. 

Verbal learning. Feigenbaum (1961) has programmed a model of 
verbal learning that he calls the Elementary Perceiver And Memorizer, 
or Epa. The program is designed primarily to account for the learning 
of pairs of nonsense syllables so that the presentation of the first member 
of a pair will elicit the second member of the pair as a response. The 
program can also be modified to treat the learning of a single series of 
syllables. The program perceives features of the incoming syllable and 
stores this information in a form that will allow quick discrimination 
of this from other stimuli but that also permits some generalization to 
similar stimuli. Associations between pairs of syllables are stored in such 
a way that experience with other pairs can partly destroy the association, 
and later experience with the original pair can reestablish the association 
on a firmer basis. The behavior of EpAm depends, not on probabilistic 
mechanisms, but on the way in which information is stored and 
processed, 
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To simulate some aspects of perception, the input stimuli are encoded 
in a large set of bits. Each letter in a three-letter nonsense syllable is 
represented by a set of at least 10 bits; a 5-bit code would suffice for 
each letter, but more bits are used so that there are several different 
ways by which one letter can be discriminated from another, and thus 
many different ways of discriminating among syllables. Epam contains a 
mechanism for generating tests that discriminate between two syllables. 
The generator selects a small subset of bit positions from the digital 
code for a syllable, and if the digits in these positions are different for 
the two syllables, the test is accepted. In the language of pattern recog- 
nition, the selected subset would 
be called an attribute, and the 
values of the bits for one of the 
two syllables would be called a 
property. The test detects the pres- 
ence or absence of the property. 

Each test can only distinguish 
between two alternatives. A new 
test is needed for each additional 
distinction. Epam distinguishes 
among a variety of syllables by 
organizing tests into a discrimina- 
tion net, which is a tree structure 
like that shown in Fig. 11-11. Each — (#) __ Discriminating test at a node 











node in the tree is a test that seeks 
Image at a terminal 


the presence or absence of some 
property. At the end of each path 
through the net is a stored image 
representing one of the syllables. 
When a syllable is presented for 
recognition, it is first subjected 


Image and cue at a terminal 
[| Empty terminal 


Fig. 11-11. A typical Epa discrimi- 
nation net. 





to the test at the top of the tree. 

The outcome will send it to one of the two nodes at the second level of 
the tree. The test at that node is applied, and the stimulus descends to 
a node on the third level. Eventually the path terminates at one of the 
stored images which is Epam’s identification of the stimulus. Whenever 
a new syllable is presented that the net cannot discriminate from a 
previously experienced syllable, a new test must be added. Actually 
IXpAM introduces some redundancy by often generating more than one 
test for a new discrimination. At any moment, the net contains tests 
for discriminating among the syllables it has experienced. The bottom 
of each path through the net is either the image of a syllable or an 
empty terminal, the latter occurring because of the redundant tests, In 
learning associated pairs of nonsense syllables, eam must first learn to 
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discriminate each member of the pair. Then, the association between 
them is formed by storing with the stimulus image a cue containing 
enough information for Epam to sort through the discrimination net 
and retrieve the response image, at the time the association is formed. 
€ince the discrimination net is continually growing, this information 
may be rendered insufficient by later additions to the net but the cue 
will be augmented by the next presentation of that stimulvs-response 
pair. 

When Epa is run on a typical paired-associates learning experiment, 
it exhibits the general characteristics of human subjects. EpaAm shows 
stimulus and response generalization, oscillation (in which a stimulus 
that once elicited a correct response later elicits a mistake), and retro- 
active inhibition. These qualitative similarities have been supported 
by quantitative comparisons in the case of the serial position effect 
(Feigenbaum and Simon, 1961). 

Binary choice. Feldman (1961) has presented a computer simula- 
tion of individual subjects who are asked to predict a series of binary 
events. This experiment has been in fashion for many years, for the 
elicited behavior is fascinating and not easily understood. In a typical 
experiment the subject must predict on each trial whether a red light 
or a green light will go on. When he has made his prediction, one of the 
two lights is turned on, according to a prearranged random sequence 
in which each light has a certain chance of being selected, e.g., 75 per 
cent red, 25 per cent green. Since the series is random, the subject’s best 
strategy is always to predict the more frequent light, but such behavior 
is rare. Instead, subjects tend to distribute their guesses in about the 
same proportion as the probabilities for the two lights. If the red light 
has a 3:1 chance of being turned on, subjects will tend to guess red about 
75 per cent of the time. This phenomenon, which has been termed 
probability matching, confounds decision theorists, who expect subjects 
to use the optimum strategy. A stochastic learning model can predict 
matching if it assumes that the subject is trying to be right on every 
trial, but it cannot explain the apparent regularities in the trial-by-trial 
choices of the subject, nor can it account for the wide individual differ- 
ences. If the subjects are interviewed after the experiment, their com- 
ments imply that they view the situation as a problem to be solved. 
Since the series is prearranged, they are unwilling to believe the experi- 
menter’s claim that the series is random. Consequently, Feldman 
decided to attempt a problem-solving model of their behavior. 

Feldman asked his subjects to give a reason for each of their choices 
and constructed a computer program that would tend to make the same 
choices for the same reasons. Each subject had several well-defined hy- 
potheses about patterns in the serics, such as “run of consecutive 
greens,” “run of reds,” “alternation,”’ and some second-order hypoth- 
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eses such as “It’s time to change patterns” or “The pattern is still going 
on, but the experimenter is trying to throw me off the track.’’ The 
program is provided with the same hypotheses and is also given pro- 
cedures for switching among them. The model does quite well: it cor- 
rectly matched one subject’s choice and his reason for that choice on 
132 out of 170 trials. Since the model was developed from the data 
against which it has been evaluated, critical judgment of its adequacy 
is difficult. 

Concept formation. Hunt and Hovland (1961) have developed an 
information processing model for the behavior of subjects in a typical 
concept-formation experiment, following the analysis of Hovland (1952). 
(See also Hovland, 1960; Hovland and Hunt, 1960.) Their program 
treats each stimulus as a set of values of specified dimensions, e.g., 
size = large, color = red, shape = circle. It keeps a record of the stimuli 
which are members of the pattern and those which are not and seeks 
to develop the rules for the concept (e.g., red or nonred) by examining 
the stored instances. The order in which the computer considers possible 
concepts has been derived from extensive experiments with human 
subjects. However, the computer could far surpass human performance 
because of its perfect memory and its persevering analysis; so limitations 
are placed on this program by restricting the amount of storage for 
past instances and the amount of processing that may be done to de- 
velop a response. Under these conditions, computer performance ap- 
proximated human performance. 

Attitude change. Abelson (1962) provides an interesting compari- 
son of a mathematical model and an information processing model 
of attitude change under controversy. The mathematical model is sim- 
pler and easier to analyze, but the information processing model covers 
more interesting behavior. In the mathematical model each person’s 
attitude, expressed on a one-dimensional scale, approaches the attitude 
of another individual in proportion to his rate of contact with that 
individual. It is assumed that the parameter for rate of contact subsumes 
the factor of persuasiveness, but negative contact is not allowed. The 
final distribution of attitude depends upon the rates of contact of each 
individual with each other individual. The rates can conveniently be 
tabulated in a matrix. Abelson shows that, if there is at least someone 
who is in contact with everybody else, either directly or through contact 
with one or more intermediaries, then eventually everyone will agree. 
In such a group, called a compact group, all the attitude scores will 
converge on a single attitude score. The speed with which the agreement 
is reached depends on a mathematical property of the contact matrix 
(the size of the latent roots) and on the initial distribution of attitude. 
But if there are mutually exclusive subgroups with no intergroup con- 
tact, then the ultimate situation is a difference of opinion, rather than 
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unanimity. There are some ways in which the model can be elaborated 
upon so that differences of opinion can remain in an initially compact 
group, but the mathematics gets unwieldy. 

Rather than tinker with the mathematical model, Abelson adopted 
the more reasonable strategy of identifying the important psychological 
processes and variables and then seeking a model to fit them. He devel- 
oped an information processing model of community controversy in 
which he is able to simulate social interactions and the effects of mass 
communication media. The model of social interactions considers each 
individual’s attitude and contacts, as in the mathematical model, and 
also considers each individual’s interest in the issue, his attitude toward 
each other individual in the social group, the particular assertions and 
arguments about the issue that are known to and accepted by him, and 
his predispositions to find certain assertions especially congenial or espe- 
cially offensive. Except for the predispositions, these factors are vari- 
ables that may change in the course of time, as the system simulation 
proceeds. The kinds of interactions that lead to changes are determined 
by a number of assumptions too extensive to record here. The informa- 
tion processing model is much more inclusive than its mathematical 
predecessor, and a computer is clearly needed to study its consequences. 

Other social processes. Coleman (1961a) has drawn attention to 
the potentialities of computers in studying social processes. The analysis 
of social structures using the methods of sociometry presents a formid- 
able data-processing task of a combinatorial nature. A computer can 
conveniently be used to separate and identify the cliques in a social 
structure such as a high-school class. Coleman (1961b) has also sug- 
gested that changes in reference groups and the general dynamics of 
social interactions can be modeled by computers, to make predictions 
about the future structure of a fluid society. 

Voting behavior has been the focus of attention of at least two sim- 
ulations (Pool and Abelson, 1961, and McPhee, 1961). Quite apart from 
the standard statistical predictions that computers make for the tele- 
vision cameras on election eve, the simulations reported by Pool and 
Abelson attempted to examine the changing behavior of blocks of voters 
over the course of a campaign. The simulations were based on the results 
of public-opinion polls taken during the national election campaigns 
of 1952, 1954, 1956, and 1958. State-by-state predictions of Kennedy’s 
strength were made for all but the Southern states, based on several 
carefully stated assumptions about the switches in votes that might 
be expected. The computer applied these rules to the many different 
categories of voters that were separately identified in the project and 
amalgamated the results into the state-by-state predictions. These pre- 
dictions corresponded much more closely with the actual voting results 
than did the results of polls taken at the same time that the simulations 
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were made. Thus a careful simulation based on an analysis of old poll 
results came closer to the mark than current but raw poll results. 
Bales (1959) has suggested the use of computers to simulate the verbal 
interactions in small groups. He and his collaborators are working toward 
a computer model that will reproduce the actual content of a small 
group discussion, once the personality characteristics of the group mem- 
bers and the discussion topic have been specified. Hare (1961) has 
described some of the progress toward that goal, including computer 
programs that simulate group decisions. It seems that a computer pro- 
gram is the best, if not the only way, to formulate a model of this kind 
of behavior. 


Problems 


1. Plan a computer program for simulating the behavior of a single neuron in a 
net, including a threshold mechanism exhibiting absolute and relative re- 
fractory phases and temporal and spatial summation. Plan a network of 
neurons in which the intensity of a stimulus is coded by both the number 
of cells that fire and the rate at which they fire. 

2. Plan a computer program for simulating any part of the nervous system 
of any organism about which detailed physiological evidence is available 
(e.g., the chemical sense of the housefly, the frog’s retina, the cat’s auditory 
cortex). 

3. Plan a pattern-detection program for sensing the presence or absence of any 
particular pattern in a 25 X 25 matrix of dots. A pattern is defined as a 
specific subset of dot positions, each of which contains or does not contain 
a dot, with a probability P;. Each of the remaining dot positions either con- 
tains or does not contain a dot, with probability Ps (see Sec. 10-5). 

4, Plan a pattern-recognition program to discriminate among several simple 
geometric shapes, e.g., square, circle, triangle. Assume that every input 
pattern is a set of filled dot positions in a 25 X 25 matrix. What provisions 
if any, must be made in your program for centering, normalizing size, elim 
inating isolates, and filling holes? 

5. Design an experiment to provide some evidence concerning the kind of input 
processing and testing that human beings use in recognizing patterns. 

6. How could a computer be programmed to detect the presence of one line 
drawing embedded in another, i.e., the Gottshalt test (see Fig. 11-12)? 


(a) (b) (c) 
Vig. Ub-12, (a) in embedded in (b) but not in (e), 
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7. The search tree for the game of ticktacktoe can be substantially reduced if 
all the symmetries are used and only nonredundant moves considered. Plan 
a program to recognize symmetric patterns in the game. (One possibility 
is to transform all patterns to a standard form.) Include a suggested computer 
representation of the current status of a ticktacktoe game. 

8. Consider a complete solution to the missionaries and cannibals riddle: 


a. List the differences that GPS should be able to notice, and indicate their 
priority. 

. List a set of operators capable of reducing differences. 

Indicate the relevance and priority of each operator for each difference. 

d. Use (a), (b), and (c) to simulate, by hand, the action of GPS in solving 
the riddle. What operations must be carried out by the executive routine 
to ensure a solution? 

e. Change the priorities and carry out the simulation again, to determine 
what differences arise. 

f. Obtain a detailed protocol of someone solving the riddle, and try to adjust 
the differences and priorities of the program so that GPS would simulate 
the observed behavior. 


oe 


9. Plan a program for solving the Tower of Hanoi puzzle. Use the GPS frame- 
work, if you wish. The puzzle consists of a set of n disks, all of different 
diameters, and three places, A, B, and C, where disks may be piled. At the 
start, the disks are all piled on A, in order of size, with the largest on the 
bottom. The object is to move the pile to B. Only one disk may be moved 
at a time, no disk may be piled on top of a smaller disk, but any disk may 
be placed in an empty place. The elegant solution has the form of a recursion. 
Split the problem as follows: Move a pile of n — 1 disks to C; then move the 
largest disk to B; then move the pile of n — 1 from C to B. Thus the problem 
can be solved for n when it can be solved for n — 1. Following this procedure, 
the minimum number of moves is 2” — 1. Presumably, a problem-solving 
program could not manage to segment the problem in this elegant way, 
unless, in some sense, the solution had been preprogrammed, but any pro- 
gram can be evaluated by comparison with the optimum solution. Consider 
instead a program that moves one disk at a time, and provide heuristics 
for choosing useful moves. Some mechanism must be provided to avoid 
endless loops. How readily will your program generalize to a problem with 
four locations for piles, A, B, C, and D? 
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Man-computer systems 


Nis information processing systems based on computers are 
. built for military and industrial applications. The processing 
in these systems is so complex that at present, and for a long time to 
come, such systems must include human operators acting as adjuncts to 
the computer. In fact, some system designers feel that fully automatic 
systems are undesirable and unattainable; they prefer to improve tech- 
niques for combining men and machines in semiautomatic systems. 
Men and machines work together in many different combinations, all 
of which may be called man-machine systems. A pilot and his sevlanhe 
form a man-machine system. So do the workers and machines on an 
assembly line, and telephone operators and their switchboards. All these 
systems may involve digital techniques in some way, but they will not be 
discussed here. This chapter is concerned with large-scale systems that 
must process information in complex ways as a basis for deciding upon 
possible actions. For the most part, the discussion will be based on 
military systems for the command and control of forces and weapons. 
The design of a man-computer system for information processing 
raises a number of difficult and provocative questions about the optimal 
use of men. How are the tasks to be divided between computers and 
human operators? What is the best way for men and computers to com- 
municate with each other? How many operators should be assigned to 
each human task? How shall the operators be selected and trained? 
Answering these questions requires a knowledge of the abilities and 
limitations of both men and machines and a thorough understanding of 
the functions that the system is to perform, Paychologists with a knowl- 
297 
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edge of computers are making substantial contributions to the design of 
man-computer systems. (See Gagné, 1962.) 

There are no well-established principles for solving the many serious 
problems that arise in particular man-computer systems. No consistent 
body of scientific knowledge exists that might form the basis of a science 
of system design. In any practical situation, the system designer must 
be guided by experience, common sense, and an appreciation of the 
problems that he faces. This chapter will discuss the major problems 
encountered in man-computer systems and will present some concrete 
examples and a few suggestions. 


12-1 Human vs. automatic functions 


The primary problem in designing a semiautomatic system is to decide 
which functions are to be assigned to computers and which to human 
operators. Clearly the computer should be assigned jobs requiring 
routine clerical labor or complicated mathematical computations, while 
operators should be assigned jobs requiring perceptual skills and mature 
judgment. But what shall be done about routine perceptual tasks or 
about tasks that seem to need hundreds of swift, mature judgments? 
A good rule of thumb is to strive for the automation of any completely 
routine job, no matter how complex the process seems. Even if complete 
automation cannot be achieved, some compromise can perhaps be 
reached in which the routine part of the task is done automatically. 
Licklider (1960) has discussed this problem, and Sec. (12-2 presents 
Yntema and Torgerson’s (1961) ideas about automatic aids to decision 
making. 

Psychologists working for the first time on a man-machine system 
may be surprised to encounter a common engineering nostrum for 
monotonous tasks, ‘Hire female operators.” The notion that women 
endure monotony better than men is dear to many engineers. It is as if 
there were a continuum between men and computers, with women some- 
where in the middle. The fact is that both men and women are poor at 
dull tasks. Any small differences that may exist between the Sexes are 
nullified by the general tendency of both to succumb to fatigue and 
boredom. Women as well as men can be expected to make errors on 
monotonous tasks that should be done by machines. 

A related fantasy is that operators become ineffective because they are 
hypnotized by their displays. While no one fully understands hypnosis, 
there is general agreement that suggestion plays a major role. In the 
case of the stupefied operator, the major factors are fatigue and boredom, 
To be sure, fatigue is not well understood either, but it is surely different 
from a hypnotic trance. To blame hypnosis is to blame the display and 
to overlook the monotony of the task. Better displays are always helpful, 
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but operators doing dull tasks will still get bored. The only sure remedy 
is automation. 

Automation is also necessary when the time scale of the activity is not 
well matched to a human operator. Actions that occur very infrequently 
and actions that must be done very swiftly are both candidates for 
automatic devices. If, for example, a counteraction must be taken 2 sec- 
onds after detecting an oncoming missile, a man would have no time fora 
considered judgment—the machine must act automatically. The ma- 
chine is not autonomous, since its rules of procedure will have been 
determined by a man, but this must be done in anticipation of the event. 
At the other extreme, detecting airplanes in the far north must be done 
automatically, for men cannot remain alert in such ‘“low-probability”’ 
situations (Jenkins, 1958). A machine could be sufficiently sophisticated 
to decide automatically whether a target is present, or it could be pro- 
grammed merely to signal the man by flashing a light that says, in effect, 
“Come have a look at this.” 

As a practical matter, a computer can replace a man only when the job 
to be performed is thoroughly understood. Or, to put it baldly, a human 
operator is often inserted in a system when the job is only vaguely 
understood, perhaps in the hope that he can figure out for himself what 
he ought to be doing. This is a convenient temporary solution, but if 
the system is really going to be used on a large scale, sooner or later 
someone will have to develop training techniques and standard operating 
procedures for the human operators on the job. The job will have to 
be analyzed carefully and completely in order to set up optimal pro- 
cedures. The analysis might even be sufficiently detailed to form the 
basis for a computer program to do the task, thus reopening the question 
of automation. Moral: If a job is too vaguely understood to be auto- 
mated, a careful job analysis must be made, after which the decision for 
or against automation can be made on more reasonable grounds. If 
2 human operator is retained, the system can be designed to give him 

adequate computer assistance. 


Illustrative example: the role of men in Sacs. The Sace (Semi-Auto- 
matic Ground Environment) system is a computer-centered air defense system. 
Radar data and other pertinent warning information are sent to the computer 
over telephone lines. Here the data from several radars are coordinated, and 
aircraft tracks are established. The tracks are compared with flight plans re- 
ceived from military and civilian airplanes, and any track without a flight plan 
is labeled as an unknown flight. If established procedures determine that the 
flight is hostile, a fighter squadron, or a BOMARC or NIKE missile, or some 
other weapon is assigned to intercept the hostile flight and the computer gen- 
erates the necessary commands to direct the interception, The flow of informa- 
tion in Saau is illustrated in Pig, 12-1, 

The Saaw system is an example of the use of men “everywhere else.” Out of 
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the total complex of operations that must be performed in an air defense system, 
the computer was assigned those tasks which were most clearly routine, clerical, 
and uncomplicated—although not necessarily simple. All other tasks were left 
to human operators. At least this was the intent. Only in the case of radar 
mapping, to be discussed below, was an operator deliberately used to do a 


@ straightforward, clerical, stupid job, because of man’s superior perceptual ability. 
2 The computer has three main functions in SAGE: tracking targets, guiding 
interceptors, and storing all relevant information about the air battle. In these 
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areas, the computer has primary responsibilities and is given some human assist- 
tance. In two other areas, track identification and weapons assignment, the 
operators have primary responsibility and are assisted by the computer. 

The Sage system has undergone many modifications since its earliest design, 
and improvements are still being made. The following account is based on the 
original version of Sager, as of 1958, with some comments about later improve- 
ments. Since this account is not up to date, and since all classified information 
has been omitted, conclusions about the current efficacy of the system should 
not be drawn from this report. 

Tracking of aircraft. The Sacu system automatically keeps a record, called 
a track, of each aircraft of interest in the surveillance area. Radar information 
is digested and correlated automatically, and the computer detects new flights 
and maintains old tracks automatically. Nevertheless, human support is needed. 
First, some initial filtering of radar data must be done to reject the large clumps 
of returns, called clutter, that sometimes occur in radar systems. In principle, 
this could be done by the computer, but the cost in time would be prohibitive. 
For a man, identifying regions having a high density of radar returns is a very 
simple perceptual task because he can see the display as a unit, whereas the 
computer would have to examine the data item by item. The process of clutter 
rejection as carried out by the human operator is called radar mapping. The 
operator uses a transparent yellow dye to paint a “map” over the clutter 
on the display, thus blocking it from the view of a blue-filtered photocell that 
is examining the scope from above. Very little intelligence is required in map- 
ping—all that is needed is the perceptual apparatus that is standard equipment 
on all human operators. 

In addition to the radar mappers, there are operators who directly support 
both the detection, or “initiation,” of new tracks and the continuance of existing 
tracks. Track initiators are operators who examine radar data to find new tracks. 
‘These men do not have the task of sensing faint radar returns. Their task is to 
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Flow of information in Sace. 
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use is made of another perceptual characteristic of the human operator. 

Track initiation could be carried out without human intervention, but, in 
areas of relatively high target density, considerable computer time and storage 
would be needed, The computer would have to consider each new radar datum 
as a potential track unless it had been already correlated with an existing track. 
Therefore, the job is split, with men working in areas of high target density 
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(generally near airports) and the computer working in areas of low density 
(generally on the periphery of the system). The split is a good one, both for the 
man and for the computer. The computer is not required to spend valuable 
time sorting through mountains of unimportant data, and the man is not re- 
quired to sit monotonously watching for very infrequent events. 

Automatic tracking also has its human support. The system designers felt 
that the machine would have trouble in some tracking situations and that a 
human operator could help out. For example, if two tracks crossed at a very 
acute angle, the computer might have trouble deciding whether they crossed 
or just touched and then parted. Or if the radar data for a track suddenly disap- 
peared, the operator might be better able than the machine to see whether the 
target had landed or had turned too sharply for the computer program to follow 
or whether, in all probability, the data were simply missing, so that the track 
should be extrapolated. The job of track monitor was therefore set up. The 
computer kept a fairly elaborate record of data acquisition for each track and 
noted the occurrence of anomalies. On that basis, it determined the extent of 
agreement between a track and its data. Whenever the data and the track dis- 
agreed to a sufficient extent, the computer called the problem to the attention 
of a track monitor. This procedure turned out to have two serious flaws. First, 
the damage had already been done. In many cases, the operator could not re- 
store the situation, because his attention was called to the trouble after the 
trouble occurred. Second, the operator had no more information than the track- 
ing program had. In the initial design of the Sacu system, the operator had no 
information, for example, about the areas in which the radar data were being 
mapped out, because the radar mapping was done in another room. Nor did 
he have access to flight plans of identified airplanes. Having no more information 
than the tracking program, and arriving on the scene late, the operator had little 
chance to improve the situation. Moreover, it was recognized in retrospect that 
the track monitor needed a fairly subtle understanding of the computer tracking 
program in order to figure out how to correct the tracking trouble. Thus, it 
turned out that track monitors had been given a difficult task without the 
necessary resources. The track monitors usually resolved their dilemma by 
letting the computer dead-reckon the track and hoping for the best. Experiments 
showed that if the computer always dead-reckoned when it got in trouble, it 
did at least as well by itself as it did with a track monitor supporting it. Ob- 
viously, there is no use putting an operator into a system to handle difficulties 
unless he is given the informational resources needed to deal with the difficulties, 

The automatic tracking program of the computer worked so well that, in 
later versions of the system, the track monitor was assigned a more passive role, 
Each monitor was assigned a number of important tracks to keep his eye on, 
and he checked their progress continually. If trouble developed, he could take 
corrective action before the situation deteriorated. 

Intercept direction. The second major function of the Sacer computer if 
intercept direction. The computer calculates values from the intercept equations 
and transmits the commands to the aircraft or other weapon automatically, 
However, in the case of piloted interceptor aircraft, the intercept director maine 
tains voice contact with the pilot in order to carry out a number of ancillary 
responsibilities, In particular, the intercept director must consider the fuel re 
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quirements of the interceptor to make sure that the airplane will have sufficient 
fuel for its return to base. When an interceptor nears an area of poor weather, 
the intercept director must override the commands of the computer temporarily 
in order to get the airplane around the weather difficulty, after which he returns 
the airplane to computer control. Another job of the intercept director is to 
act as track monitor for the interceptor under his control. It turns out that he, 
rather than the actual track monitors in the system, has the necessary informa- 
tion in order to resolve any tracking trouble that may arise in automatically 
tracking interceptors. Thus the intercept director is a busy fellow, although, 
according to the initial concept of the task, his job was simply to give assistance 
to the computer in performing an essentially automatic task. 

Information storage. Most information from external sources goes directly 
into the computer without human intervention, but in a few cases, such as 
weather reports, the information must be recoded manually before the computer 
can accept it. The recoding is essentially a clerical function, but the information 
must be read from charts and maps by men, because computers cannot yet 
read printed matter, let alone interpret maps. Pattern recognition is an inherent 
part of the task, and as yet only human beings have this ability. 

Track identification. In Saas, track identification is handled manually, with 
some computer assistance. The assistance consists mainly in providing relevant 
information about a new track to the identification operator for his consideration. 
As each new track appears in the system, it is automatically displayed to the 
identification operator, together with all the flight plans that might conceivably 
match the track. Since the rules for matching incoming tracks with flight plans 
seem quite precise, the computer could do much more than pick the likely 
flight plans. It could automatically make an identification based on the rules 
and call on the operator only in cases where more than one flight plan correlated 
with a track. With automatic weapons assignment, the computer might then 
automatically release an antiaircraft missile if a track were identified as hostile. 
In practice, of course, a human decision is required, because it is considered 
essential not to shoot down a friendly airplane even if it does not quite meet the 
rules of matching. That is, the identification rules are actually much less precise 
than they seem. Many other factors are involved in not very specific ways, so 
that no definite identification procedure can be precisely specified for all occa- 
sions. Hence the need for a human operator. 

Weapons assignment. An important job that is almost exclusively a human 
function is that of weapons assignment. Here the computer merely provides 
relevant information upon request to the weapons assigner. The actual assign- 
ment of weapons to hostiles is entirely manual. All the decisions are man-made, 
and all the procedures necessary to carry out those decisions are initiated by 
man. Weapons assignment requires more complex decisions than are involved 
either in tracking or in intercept direction. It is easy to understand why, in the 
initial SAGE design, weapons assignment was prudently left to human operators. 
But as our understanding of man-computer systems progressed, it became rea- 

sonable in later versions of Saaw to attempt semiautomatic weapons assignment. 

One of the serious difficulties in weapons assignment arose because the job 
was too big for one man, Initially, the job was split among four weapons directors 
not located next to each other, but all presumably coordinated by one senior 
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weapons director. Since weapons assignment is essentially a unitary problem, 
the necessary communications among the men took up an appreciable amount 
of their time. Later versions of the system provided for face-to-face communica- 
tion among the assigners, which seems very desirable whenever a single task 
must be divided among several men. 

This brief look at the Sacr system exemplifies many of the functions that 
can be assigned to human operators in a semiautomatic system, from simple 
perceptions to delicate judgments. In Sacx’s day there was no alternative to a 
human operator for these functions. Now that computers are acquiring new 
talents, they may compete with human operators for these jobs, and matters 
of efficiency and cost will be considered in choosing between men and computers. 


12-2 Computer assistance in decision making 


Yntema and Torgerson (1961) have suggested that a man could obtain 
computer assistance in making decisions by telling the computer how he 
wants decisions made and letting the machine make the decisions for 
him. They state the problem lucidly: 


High-speed digital computers can be programmed to process large amounts 
of information quickly and to make decisions very rapidly, but they have not 
yet been made to show more than the first glimmerings of intelligence. In the 
present state of the art, they certainly cannot he programmed to display the 
generalized sort of good judgment called common sense. Thus the designers 
of a large control system usually conclude that some of the decisions the 
system must make are too important or too complicated to be entrusted to a 
machine. Such decisions are left to a man who acts as a sort of deputy planner, 
filling out the design of the system when contingencies arise that could not 
be included in the computer program. 

A perennial problem in such cases is how to combine the machine’s speed 
and the man’s flexible good sense without sacrificing too much of either, 
One typical solution is to have the machine assemble and present to the man 
the facts he will probably want in reaching a decision, allow him to call for 
additional information if he thinks he needs it, and require him to indicate 
to the computer the action on which he decides. Another is to let the machine 
make the decisions according to the comparatively simple rules that can be 
programmed into it, but require the man to monitor the situation and counter 
mand the machine’s orders if he finds them too foolish. Either way, the man 
must examine one by one all of the situations calling for judgment. He must 
digest the information presented to him, perhaps ask for and digest’ more 
information, reach his decision, and perhaps.communicate it to the machine, 
This takes at least a few seconds, which is a long wait by computer standards, 
And the seconds can pile up if a number of actions must be taken in quick 
succession. Unless the system has been shrewdly designed, a sudden flurry 
of decisions may cause the man to fall behind. He may then become a bottles 
neck, or be so hurried that he behaves foolishly himself. 
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Watching an operator struggle with a crisis like that gives one the impression 
that man and computer are not complementing each other efficiently. The 
inefficiency seems particularly obvious when, as is often the case, the man 
appears to be doing the same job over and over. The decisions are all slightly 
different, but the factors he considers in making them are the same. He is in 
effect using the same decision rule again and again—a rule that takes account 
of all the special facts and temporary considerations that are known to him 
but are not recognized by the machine. One feels that he and the computer 
could cooperate better if he could tell it what his temporary rule was. He 
would then be free for other tasks, and the machine would be free to make 
the decisions at its own speed. 

That, we suggest, is precisely what will have to be done in many cases. 
In the next few years, real-time systems will be required to make decisions 
calling for even more judgment than is now needed, more speed will be re- 
quired, and the number of decisions will be larger. One might say that the 
sheer volume of good judgment demanded per minute will increase. Hither 
the number of men will have to increase too—which rapidly makes a system 
unwieldy—or some of the decisions will have to be mechanized. Until com- 
puters acquire a little common sense, mechanization implies that people will 
have to direct the machines by choosing the rules according to which the 
decisions are made. 

All this may sound more dramatic than we intend. The early techniques 
for telling a computer how to make decisions will probably involve approxi- 
mations and be subject to a certain amount of error. So the first decisions 
to be mechanized will usually not be the large, spectacular ones that demand 
the utmost delicacy of judgment. Probably the computer will at first be used 
to back up the operator in lesser situations where prompt action is desirable. 
From time to time, he will give the machine approximate instructions that 
will enable it to take some fairly sensible action at once, instead of just waiting 
idly when he falls behind or is busy with other tasks. Then too, there are 
always a number of routine, time-consuming problems that have no direct 
bearing on the output of the system, but still require a certain amount of 
common sense. How the system should distribute the data-processing load 
among its own parts is one example. Mechanizing these smaller decisions will 
help keep the men free for the larger ones. 

So far we have been discussing real-time systems, where the emphasis is 
on speed. Distilling an expert’s judgment into a computer may prove equally 
important in long-range planning. In logistics and in business it is now common 
to use computer. programs that calculate how to maximize some quantity 
or other; linear programming is the prime example. As the mathematics 
become more sophisticated and computers become more powerful, the success 
of these methods will depend less and less on the number of variables and 
the complexity of the relations that can be considered. Planners will find 
themselves paying even more attention than they do now to the precise defi- 
nition of the quantity to be maximized. And here expert judgment is necessary. 
What is the trading relation between cost in dollars and the speed with which 
orders can be filled? And how are both of these factors to be balanced against 
the nonmonetary disadvantages of more employees? One way of dealing with 
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such questions will, we anticipate, be to get an expert to summarize his 
experience and good sense in rules that define with acceptable accuracy the 
worths of all possible plans. The form of the rules must be such as to admit 
of a practical algorithm for selecting the best solution from the infinity of 
alternatives typical in these planning situations, but otherwise the problem 
is not too different from that which will be faced in real-time systems. We 
assume that many of the psychological techniques developed for one appli- 
cation will be useful in the other. 


Yntema and Torgerson go on to suggest that many human decisions 
are simpler than they appear. In a complicated situation that involves 
many factors, the human decider is likely to ignore the minor factors 
and consider only three or four major factors. Moreover, the interactions 
among these factors are often negligible. Most people, if asked to explain 
how the various factors influence their decision, will report that each of 
the factors depends on all the others. But, in some experiments, people 
behave as if they were evaluating the factors independently. 

Consider the problem of choosing between two alternatives. Suppose 
that only three factors are important in making the choice. Each of 
these factors is assumed to have a number of levels. For example, if a 
research scientist is choosing between jobs that vary on the factors of 
salary, quality of coworkers, and tenure, the factor of tenure might have 
the levels permanent, normal, and 2-year contract. An alternative is 
characterized by its level on each of its factors. A particular job might 
pay $10,000 per year and guarantee permanent tenure, but the quality 
of one’s coworkers might be second-rate. Each alternative has a worth 
that, in the most complex situation, is determined by its particular 
combination of levels on the three factors. Every distinct combination 
has a distinct worth. Nevertheless, it is often possible to evaluate the 
worth of each factor separately and to obtain an approximation of the 
total worth of an alternative by summing the contributions of the sepa- 
rate factors, the interactions among factors being ignored. In the job 
example, it is not necessary to consider what effect permanent tenure 
has on the worth of colleagues or salary. If the scale of worth of each 
separate factor is related monotonically to the total worth of a job—i.e., 
if the worth of a job increases with higher salary, better colleagues, and 
longer tenure—then the sum of the separate worths is an excellent 
approximation to the total worth. Indeed, it seems to be a mathematical 
fact that interactions cannot possibly have very much effect in this case, 

Yntema and Torgerson discuss how this approximation might be used 
in practice: 


Consider an example of the sort that arises in the tactical control of forces, 
Let us make the improbable assumption that a computer is to be used in 
dispatching forces to put out forest fires, The firefighters are stationed at 
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bases scattered about in the forest, and it is considered impractical to move 
them from one base to another to fill the gap left by a force that has gone 
to a fire. The computer has a record of the current amount of force at each 
base. When a new fire is reported the time required to get there from any 
given base can be calculated automatically. 

The bases are then the alternatives among which a choice must be made, 
and the worth of each depends on the time required to get to the fire and 
on the way the remaining force would be distributed if that base were chosen. 
This looks like a formidable problem. Evaluating a distribution of forces 
sounds like one of those complicated tasks that can be performed by a person 
but cannot easily be explained to a computer. 

Consider, however, the factors that determine the worth of a tentatively 
chosen base. If there are n bases, then there are n + 1 factors: one for speed, 
and one for the amount of force that would remain at each base, including 
the chosen one. More speed would always be desirable, and other things 
being equal, any situation would be improved by the addition of more force 
at any base. In other words, the worth of a tentatively chosen base increases 
with all of the factors. Thus the computer would probably make a fairly good 
decision if it were provided with the average worth of each delay in getting 
to a fire and the average worth of each level of force at each base... . 

Suppose next that the commander were informed that part of the forest 
had become particularly dry. If he were making the decisions himself he would 
try to conserve force in that region by choosing other bases whenever he had 
a reasonable choice. He could make the computer behave in much the same 
way by telling it to weight especially heavily the worth of force at bases 
in the danger area. This illustrates one way of taking account of conditions 
known to the man but not to the machine: it will sometimes be possible to 
express the influence of outside information by weighting the main effects 
of the factors considered by the computer. 


In practice, a computer that makes decisions by combining the worths 
of separate factors, according to weights supplied by the man, is likely 
to do as well as the man would have done. In cases when the worths of 
alternatives are quite different, the machine will make the same choices 
as the man, even though its approximations to the worth of the al- 
ternatives may be slightly inaccurate. Differences between the man and 
the machine will tend to appear only when the alternatives are of nearly 
equal worth. Of course, the machine always knows how close the 
decisions are and can present the difficult ones to the man, but if the 
worths are nearly equal, then any decision is as good as another. If a 
man is presented with the same alternatives on successive days, he often 
fails to agree with himself when the decisions are close. In one laboratory 
experiment, Yntema and Torgerson found that the man disagreed with 
himself almost as much as the computer disagreed with the man’s 
choices, Obviously, much research remains to be done before a computer 
starts making important decisions, but the possibility is most attractive, 
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12-3 Division of responsibility 


Having decided what functions are to be performed by human operators, 
the system designer must next divide the human functions among a num- 
ber of operators so that each man has only a man-sized job. Normally 
each function is assigned to one or more operators, but this is not neces- 
sarily the best arrangment. It might be better to assign each incoming 
item to a single operator who must perform all functions for that item. 
Thus in Saas, instead of having track initiators, track monitors, iden- 
tifiers, intercept directors, etc., all the operators might be capable of 
performing all the functions. Each new target might be assigned to an 
operator whose job is to shepherd that target through all phases of the 
system. To oversimplify, we could call the former arrangement a series 
system and a latter a parallel system. The parallel system would not be 
practical in Sage, but it is the common arrangement in a restaurant. 
Each waiter is assigned several parties. He serves all courses to his 
parties. Once in a while there is a specialist, like a wine steward, but one 
never finds a soup waiter, a salad waiter, an entrée waiter, and a dessert 
waiter. Only in a cafeteria is the series arrangement encountered. Of 
course a pure system is seldom encountered anywhere. In the restaurant 
the hatcheck girl and the bus boy are specialists, and during the off-hours 
the cafeteria may have only one or two attendants to serve all dishes. 

Problems in assigning responsibility are not unique to man-computer 
systems. All sorts of human organizations, from business offices to 
departments of the Federal government, have similar problems. Students 
of human organizations talk about ‘“departmentalization” and the 
“assignment problem” but not much is known about them, despite the 
pervasiveness of the problems. A recent review of the literature as it 
relates to human organizations can be found in March and Simon (1958). 

In practice, there are advantages and disadvantages to both the 
parallel and series arrangements. In Saag, for example, a track monitor 
might need information available only to the intercept director, and vice 
versa. But in a parallel SacE organization, each operator would again 
need information from other operators—e.g., are these tracks likely to 
intersect? The major disadvantage of the parallel operation is that each 
operator must understand all the functions of the system; the major 
advantage is coherence of processing for each track. The results of an 
unpublished study of a manual air defense system, done at the Lincoln 
Laboratory, suggested that a parallel system was better when the 
operators were sufficiently intelligent to take advantage of its flexibility, 
The study also suggested that face-to-face communication was im= 
portant for the successful operation of a parallel system. 

Queuing. When a particular human job has been defined in the 
system, the system designer must determine how many operators should 
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be assigned to perform this job. For example, the question arises of how 
many track monitors or how many track initiators are needed in the 
SaGE system to cope with the expected traffic. It is inadequate, indeed 
it is a bad mistake, to determine the mumber of operators required by 
merely dividing the expected average traffic by the average number of 
items that one operator can handle. In almost all situations, the traffic is 
not constant but varies from time to time. If the operators are working 
full time in order to keep up with the average amount of traffic, then the 
system will become overloaded whenever the traffic load increases above 
the average. In fact, it can be shown that the system will tend to be 
overloaded even when somewhat less than the average amount of 
traffic exists, because of momentary fluctuations. To the extent that 
fluctuations in traffic density occur regularly, extra operators can be 
assigned during times of peak traffic, but the moment-to-moment 
fluctuations in traffic density require more basic consideration. 

The problem is well known in the design of telephone exchanges. 
Enough dial equipment or enough manual operators must be provided 
to take care of all calls as they originate, without, at the same time, 
putting in any more equipment or any more operators than is absolutely 
necessary. If we assume that each operator can handle one call at a time, 
consuming a fixed amount of time per call, and that calls originate at 
random time intervals, we can calculate how long a call must wait, on 
the average, before it is serviced, as a function of the number of operators 
and the average number of calls. Moreover, we can calculate the sta- 
tistical distribution of service times and can determine how many oper- 
ators are required so that no more than a small percentage of the calls 
must wait more than a small time interval, e.g., no more than 1 per cent 
wait more than 10 seconds. When more calls occur than can be serviced 
immediately, a line, or queue, must form. We can also calculate the 
average length of the queue. 

In the context of an information processing system, it is convenient 
to talk about messages occurring and being processed by operators. 
In the simplest case, it is assumed that messages occur at random time 
intervals, independently of each other, with a certain average rate, say, 
r. Suppose further that each message can be serviced in ¢ sec. For any 
combination of r and ¢ and for each number of operators, we can de- 
termine the average waiting time for a message, the average length of 
the queue, and the proportion of messages that have to wait more than 
a certain fixed time interval. 

Very often the problem is more complicated. There may be several 
types of messages each occurring at a different rate. Further, the oc- 
currence of one message may imply the later occurrence of follow-up 
messages, The service times for each type of message may differ, and 
service times cannot realistically be treated as constant, since they are 
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subject to random variations. Mathematical analysis of such complex 
queues is very difficult, and Monte Carlo techniques must be employed 
to determine the properties of the queues (see Chap. 8). 

Some idea of the effect of arrival rate on waiting times can be obtained 
from Fig. 12-2, which shows the probability of a wait of at most ¢ time 
periods for the simplest type of queuing process: each item takes a 
constant time to be processed, there is only one processor, items are 
processed in a first-come-first-served queuing discipline, and items to be 
processed arrive at random times so that the time intervals between 
successive arrivals have an exponential distribution. For convenience, 


1.00 


0.80 


0.60 


0.40 


Probability of a wait of at most ¢ 


0.20 





0 2 4 6 8 10 12 14 16 18 
Time in processing units, t 
Fig. 12-2. Empirical waiting-time distributions as a function of load, a. Each 
curve is based on a sample of 125,000 waits. 


time is measured in units of the constant processing time. The parameter 
a specifies the average rate of arrival of items per processing time unit, 
The curves show that, as a approaches 1, the waiting times become 
longer and longer. The probability that any incoming message will have 
to wait no more than five processing time units is 0.94 for a = 0.75 but 
is only 0.45 for a = 0.95. 

The empirical curves in Fig. 12-2, which are based on a sample of 
125,000 waits, were obtained by Smith and Klem (1959) in the course 
of a study on the reliability of empirical waiting-time distributions, 
They compared these curves with many equivalent distributions based 
on only 4,000 waits, to determine the maximum deviation to be expected 
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in the smaller samples. They found that the deviations were greater for 
values of a near unity. Some typical results can be summarized as 
follows: When a was 0.75, the estimates of the probabilities of various 
waiting times were quite close to the true probabilities. Only 10 per cent 
were off by more than 0.05. But when a was 0.95, 10 per cent of the esti- 
mates were off by at least 0.20. Thus, when a is close to 1.0, small samples 
can give poor estimates; much larger samples must be used to get good 
Monte Carlo estimates of the waiting-time probabilities. 

An information processing system that expects input messages at 
random times and tries to have barely enough operators for the average 
load (a nearly 1.0) can expect long waiting times. Moreover, Smith’s 
results show that a few hours of system tests may not be enough to 
determine the capacity of the system. Extensive tests are needed. When 
a system is complex and has many operators, there may be no way to 
know what a is or whether or not there is any regularity in the arrival 
times of messages. If there is some chance that a may be near 1.0 and 
that arrival times may be random for a part of the system, then a few 
system tests cannot be relied upon. Extensive tests or extensive Monte 
Carlo simulations are in order. 

Queuing problems are crucial to many kinds of systems and have 
prompted many theoretical studies. Feller (1957) and Morse (1958) give 
good general accounts. 


12-4 Displays and controls 


More psychological research has been done on the problem of displaying 
information to the operator and providing him with suitable controls 
than on any other man-machine question. Many of these studies have 
been concerned with the design of characters and symbols to be used in 
expressing the information, in particular ways of coding complicated 
information by a few symbols, and in comparing different types of 
controls for making indications to the computer. These straightforward 
human engineering experiments will not be discussed here (see Chapanis, 
Garner, and Morgan, 1949; Woodson, 1954; or Baker and Grether, 
1954). 

A computer can generate displays on a variety of different devices that 
are more versatile than the standard radar displays of precomputer days. 
A computer-controlled cathode-ray tube is the fastest and most versatile 
display device. The details of CRT displays are discussed in Sec. 10-7. 
Other display mechanisms are also useful. Simple signal lights can 
transmit binary information, such as “input error’ or “new target 
detected.” Often the light illuminates a message so that the man need 
not remember the meaning of the light itself, Mlashing lights or buzzers 
can be used to call the operator's attention to a crisis, Tape recordings 
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of warning messages, spoken by a woman, have been used to alert pilots 
to an emergency—e.g., ‘Your fuel supply is low.” For less immediate 
communication, automatic typewriters, printers, and plotters can be 
used to send messages to the operator. 

Operators generally communicate with computers by push buttons, 
pointers, and keyboards. Push buttons can be used in common, pre- 
arranged situations. The button, which can be labeled to remind the 
man of its function, indicates a particular choice in a prescribed situ- 
ation. When the operator wishes to point to some target on his CRT 
display, he can use one of the pointing devices described in Sec. 10-7. 

Sometimes the operator must put large amounts of tabular infor- 
mation into the computer. In doing so he must generate a message for 
the computer, rather than just pushing one of a few alternative push 
buttons. Some sort of keyboard is provided, on which the message is 
typed. The keyboard may be attached directly to the computer, or it 
may simply punch a tape or card which must then be put into the com- 
puter. Fancier schemes allow the operator to set up a complete message, 
in the manner of a typesetter, so that he can scan it for typing errors 
before sending it to the computer. At present, such messages must be 
composed in a highly stylized language, and rigid formats must be 
followed to define exactly what information is being sent. The possibility 
of communicating with a computer in natural language is discussed in 
Chap. 13. The further possibility of voice communication is under study 
(see Sec. 11-3), but a practical voice-communication system is many 
years in the future. 

Whatever communication procedure is used, the operator must be 
provided with immediate feedback concerning the actions he takes. It 
is of vital importance that any push-button action or indicated point on 
the display should result in some change in that display, so that the 
operator can see that his action made a difference to the system. This 
feedback should occur as soon as possible after input to the computer, 

While the many different display and control devices may provide 
very flexible man-computer communication in large systems, they also 
complicate the basic problem of what information to display to a par- 
ticular operator and what particular controls to give him. Here the 
system designer faces a serious dilemma. Since the man is in the system 
to perform those operations which cannot readily be automated, it would 
seem to follow that he should be given all possible resources for perform 
ing his job. All the information in the computer that is conceivably rele« 
vant to the particular job should be available to him, and he should be 
able to take a large variety of possible actions. If this philosophy is 
followed in practice, however, the man is very quickly swamped with 
information. To avoid inundating the operator with information, the 
designer would like to present only the information that the operator 
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needs at the moment. However, it is axiomatic that the needed infor- 
mation cannot be determined a priori, for if the designer could determine 
just what information is needed by an operator, the operator’s job could 
probably be done automatically. Apparently, therefore, the operator 
must have control over the information that he receives and be able to 
call for various kinds of information as he finds a need for it. Yet valuable 
time can be lost if the operator must call for several different pieces of 
information sequentially, in order to make a decision. The problem, then, 
which is largely unsolved, is how to provide the man with instantaneous 
access to any information he may need without cluttering his display. 

Experience suggests that the display problem is a system problem. 
Determining what information should be displayed to an operator 
requires a thorough analysis of his job and its relation to the rest of the 
system. Analyzing the job of an operator requires a detailed list of what 
he is expected to do in each kind of situation that might arise. Such a 
list is not obtained without considerable sweat and tears, but once made, 
it can form a logical basis for designing the displays and controls. With- 
out such an analysis the designer is forced to rely on his own judgment 
and intuition, which can be very misleading. Displays and controls 
cannot be considered as separate components to be designed inde- 
pendently of the system but must be an integral part of the system 
design. They must be tailored to the demands of the jobs in the system, 
and the system must be capable of producing the needed displays and 
reacting swiftly to the control settings and other human communi- 
cations. 


12-5 Manning automatic systems 


It might seem that the era of automation will release so many men from 
jobs that there will be plenty of manpower. However, there is reason to 
believe that the manpower situation is really made worse instead of 
better by automation. Automation puts increased demands on the men 
who maintain the equipment and creates the need for additional high- 
caliber personnel to serve as computer programmers. Present experience 
also suggests that automation does not always reduce the number of 
operators. Sometimes it increases the capability of the same number of 
operators. Moreover, automation increases the complexity of the human 
jobs in the system. Therefore there is every reason to expect increased 
competition for the most highly qualified individuals, of whom there is 
only a limited number. 

Maintaining a large computer-centered system requires a corps of 
highly trained technicians ready to repair any malfunction at a moment’s 
notice. In a highly interconnected system, detecting the presence of an 
equipment failure is often difficult, and tracing the source of the trouble 
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requires an expert diagnostician.! Maintenance woes can be alleviated by 
a continuous program of preventive maintenance, in which potential 
failures are sought before they actually occur. Proper design can also 
help, though little things are sometimes overlooked. The author once 
encountered a design for an operator’s console which included about 100 
small lamps to transilluminate the plastic labels on the push buttons and 
display controls. The lamps were very difficult to replace, and the de- 
signer had unfortunately chosen to use a lamp with a rated lifetime of 
only 500 ‘hr. On the average, then, one lamp could be expected to burn 
out on the console every 5 hr, and a system using many consoles would 
need several men working full time replacing lamps. To alleviate the 
problem, the life expectancy of the lamps was greatly increased, with 
little loss in illumination, by simply using a lower voltage than the rated 
voltage of the lamps. 

Both semiautomatic systems and so-called “fully automatic” systems 
require large numbers of programmers to keep the operating programs 
up to date as changes are made in the equipment and in operating 
procedures. While the equipment in a computer system is relatively 
permanent, the computer program is dynamic. Nearly as many pro- 
grammers are needed to maintain it as to write it in the first place. 
Programming the Sacer system was such a big task that it was partly 
responsible for the organization and growth of the System Development 
Corporation, which now has the task of providing and maintaining the 
computer programs for the Sace computers. Future systems more 
automatic than Saar are likely to require even more extensive computer 
programming in support of the system. At the present writing, no good 
selection tests are available for choosing good programmers. The 
psychological study of the programming process is a fertile, untapped 
area for research. 

The selection, placement, and training of operators for the system 
remain of vital concern. Standard personnel-selection tests can be used, 
but the sorts of abilities required of operators may require new tests. 
Automatic systems like Sacre emphasize human perceptual skills and 
mature judgment, rather than clerical and numerical abilities. A high 
level of verbal and quantitative aptitude is required of operators who 
must understand the computer program in order to do their jobs, 
Because programmers and maintenance men also need these aptitudes 
the real personnel problem is not so much selection as placement. Jobs 
are competing for men rather than the reverse, presenting a formidable 
problem in the optimum utilization of available manpower. 

System training. A new operator in a man-computer system not 


1 The human problems in maintenance have been discussed at length in a special 
issue of the IRE Transactions on Human Factors in Electronics, September, 161, 
vol, HEE 2, no, 2. 
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only must be taught the basic skills involved in his job but also must be 
given considerable experience in an ongoing system. What he does in 
his job will affect the system and will therefore affect what other persons 
in the system can do. Conversely, his actions will depend, to an extent, 
on the actions of others in the system. Experience in the interactions 
among operators can be achieved only by operating the system as a 
whole. It is obviously unwise in military systems to depend on the day- 
to-day operation of the system to provide this experience. Not only 
should the operators be thoroughly trained before they begin operational 
tours of duty, but the everyday experience cannot be expected to provide 
the necessary background for possible crises. Instead, simulated exercises 
are prepared, and the entire system is exercised. 

Preparing an exercise for a large man-computer system is a very com- 
plicated process. A large number of inputs to the system must be devised 
and properly coordinated so as to simulate a real-life situation. More- 
over, a realistic situation cannot be planned completely in advance, since 
its course of development should obviously be influenced by the actions 
of the personnel undergoing training. To provide for such contingencies, 
a complex computer program is needed to run the entire simulation in 
conjunction with the main system program. Some operators are also 
needed to simulate the action of the ‘“‘enemy.” Thus the training of the 
system becomes a man-computer supersystem, involving both a com- 
puter program and human operators. The supersystem must include 
provision for evaluating the actions of the system being trained and 
should provide immediate feedback to the operators so that they will 
know when they are doing well and when they are doing poorly. Pro- 
viding adequate training is thus a big job, requiring many full-time 
people, and thus adds to the list of extra personnel needed by this 
“automatic” system. 


12-6 The man-computer system in the laboratory 


‘There is very little scientific information about man-computer systems. 
Most of the opinions and attitudes expressed in the present chapter are 
based on anecdotal evidence or on very small samples. Presumably, a 
scientific study of man-computer systems is possible. At least the 
properties of a particular system can be examined in a laboratory setting. 
The laboratory study of a system differs in essentially no respect from 
systems training. The entire system is operated on simulated situations, 
and information about the actions of the system is recorded. 

One vital problem that can be studied in the laboratory is the reaction 
of the system to an overload, Will the system break down completely 
when it has more information than it can handle, or will it deteriorate 
gracefully? A series of experiments at the RAND Corporation's Systems 
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Research Laboratory attempted to answer this question for a manual 
air defense system. The simulated traffic load was steadily increased, 
and the reactions of the system were recorded. The major conclusion 
was that the system scarcely deteriorated at all. Rather, the crew 
streamlined their procedures and managed to keep up with the increased 
load. Further increases led to further streamlining (see Chapman et al., 
1959; Sweetland and Haythorne, 1961). Similar effects have been seen 
in other systems. Experiments of this kind are currently being performed 
at the Systems Development Corporation. 

Alternative system designs can sometimes be evaluated compara- 
tively in the laboratory. Parallel and series organization of a manual 
air defense system were compared in an unpublished study at the Lincoln 
Laboratory, resulting in the conclusions mentioned above in Sec. 12-3. 
RAND’s Logistics Research Laboratory has been used to compare vari- 
ous systems for logistics support of aircraft maintenance (Haythorne, 
1960). Air-traffic control systems have been studied at the Laboratory 
of Aviation Psychology at Ohio State University (Kidd, 1959). 


Problems 


1. A certain air-traffic control system requires input messages about flight plans 
to have the following format: 


Flight number 6 characters 
Place of departure 3 character code 
Estimated time of departure 4 numerals 
Destination 3 character code 
Estimated time of arrival 4 numerals 
Requested altitude 2 numerals 


Any unused characters must be filled by X’s. The flight plan for a hypothetical 
American Airlines flight 132, from the New York Idlewild Airport to the 
Los Angeles International Airport, leaving at 8:30 a.m., arriving at 1:30 
P.M., and requesting an altitude of 28,000 ft, would result in the following 


message: . 
AA132XIDL0830LXA133028 


a. Diagram a program to read in such a string of characters and form & 
six-register record of the data. The program should check the location 
codes against a library and tilt (give an error indication) if a code in the 
input message does not agree with any stored code. The time characters 
should also be checked for reasonableness. 

b. Assume a typewriter input-output system, and plan a system for calling 
the’ operator’s attention to any detected errors. Arrange some method 
for him to correct the error without having to retype the whole message, 
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c. Plan a program that will accept a message with an arbitrary number 
of blanks between items. 

d. Design a procedure for making changes in an initial plan without retyping 
the entire plan. 


2. Consider the problem of setting up an automatic library. Include provision 
for push buttons and keys for standard questions or requests for books. 
At what points would human librarians be needed? 





13 


Communicating in natural language: 


the Baseball program 


EN typically communicate with computers in a variety of arti- 
M. ficial, stylized, unambiguous languages that are better adapted 
to the machine than to the man. For convenience and speed, many 
future computer-centered systems will require men to communicate 
with computers in natural language. The business executive, the military 
commander, and the scientist need to ask questions of the computer in 
ordinary English and to have the computer answer their questions 
directly. The present-day need for a programmer to act as a go-between 
discourages many from using computers in situations where automatic, 
efficient data processing is badly needed. The intermediate programming 
process, with its inevitable debugging phase, often takes several days, if 
not weeks, unless the query has been anticipated and a program pre- 
pared in advance. When computers acquire the ability to interpret 
natural English, as well as to retrieve and process data in flexible ways, 
they will be able, without the help of an intermediary, to answer novel 
questions in a matter of minutes. The resultant increase in the power of 
military command and control systems will have a significant bearing 
on our defense stature, and the advantages for business and science are 
equally exciting to contemplate. 

The dream of a computer that can respond intelligently to natural 
English inputs is not completely fantastic. Some progress has already 
been made, in the form of computer programs that can answer a limited 
variety of questions about a limited subject matter. McCarthy (1959) 
described the Advice Taker, a program to receive categorical statements 
and to answer questions by drawing conclusions from the statements, 
Lindsay (1960) wrote a program that abstracts information about 
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familial relationships from statements like “John is Mary’s brother” 
and generates a family tree depicting the relationships. Phillips (1960) 
wrote a program in Lisp (see Chap. 5) to store simple English sentences 
and to answer questions phrased in English about the subject, object, 
verb, place, and time of the stored sentences. His program used a dic- 
tionary that associated words with parts of speech in order to make a 
simple syntactic analysis. To answer the question “Where does the 
teacher read books?” Phillips’s program finds the stored sentence ‘The 
teacher reads books in the classroom’’ and answers “In the classroom.” 
Simmons et al. (1961) described a program called Synthex for doing the 
same sort of thing in a larger context. Kellogg (1960) has described a 
general program for information retrieval. General problems in the auto- 
matic processing of natural languages are discussed in Garvin (1962). 

Green, Wolf, Chomsky, and Laughery (1960) developed a program 
called Baseball that operates in the context of baseball scores and can 
answer such questions as ‘Did the Red Sox beat the Yankees five times 
in July?” and “How many games did the White Sox win by one run?” 
The data are the month, place, day, teams, and scores for each game in 
the American League for 1 year. In this limited context, a small vocabu- 
lary is sufficient, the data are simple, and the subject matter is familiar 
to most Americans. The Baseball program has important implications 
for man-machine communication as well as the psychological analysis 
of human communication; so an abbreviated account of the program is 
appropriate for this book. 


13-1 The organization of the Baseball program 


The Baseball program consists of two parts. The linguistic part reads 
the question from a punched card, analyzes it syntactically, and deter- 
mines what information is given about the data being requested. The 
processor searches through the data for the appropriate information, 
processes the results of the search, and prints the answer. Associated 
with the linguistic part of the program is a dictionary containing the 
definitions of all words to be encountered in the input questions. Asso- 
ciated with the processor is a set of data containing the information upon 
which the answers to the questions will be based. 

The program is written in Ipt-v, an information processing language 
that uses lists and hierarchies of lists, called list structures, to represent 
information (see Chap. 5). Both the data and the dictionary are list 
structures in which items of information are expressed as attribute- 
value pairs, e.g., Team = Red Sox. 

Specification list. Fundamental to the operation of the Baseball 
program is the concept of the specification list, This list can be viewed 
4s & canonical expression for the meaning of the question; it represents 
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the information contained in the question in the form of attribute-value 
pairs. The specification list is generated from the question by the lin- 
guistic part of the program, and it governs the operation of the processor. 
For example, the question ‘““Where did the Red Sox play in July?” has 
the specification list: 


Place =? 
Team = Red Sox 
Month = July 


Some questions cannot be expressed solely in terms of the main attri- 
butes (Month, Day, Place, Team, Score, and Game Serial Number) but 
require some qualification or modification of these attributes. For ex- 
ample, on the specification list of ‘How many games did the Red Sox 
win in July?”, the attribute Team is qualified by Winning, and Game 
is modified by Number of, yielding: 


Game (number of) = ? 
Teamvwinning) = Red Sox 
Month = July 


Data. The data are organized in a hierarchical structure, like an 
outline, each level containing one or more items of information. Rela- 
tionships among items are expressed by their occurrence on the same 
list or on associated lists. The main heading, or highest level of the 
structure, is the attribute Year. At the moment, all the data are for the 
year 1959, but the possibility of further years is indicated by including 
this attribute in the data. Below the year on the next level is the attri- 
bute Month. For each month, the data are further subdivided by Place: 
Below each place under each month is a list of all games played at that 
place during that month. The complete set of items for one game is 
found by tracing one path through the hierarchy, i.e., one list at each 
level. Each path contains values for each of the six attributes, e.g.: 


Year = 1959 
Month = April 
Place = Boston 
Game = 54, Day = 29 
(Team = Red Sox, Score = 5) 
(Team = Yankees, Score = 3) 


The parentheses indicate that each team must be associated with its own 
score, which is done by placing them together on a sublist. , 

The processing routines are written to accept any organization of the 
data. In fact, they will accept a nonparallel organization in which, for 
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example, the data might be as above for all games through July 31 and 
then, for the rest of the season, organized by place, with month under 
place. The processing routines will also accept a one-level structure in 
which each game is a list of all attribute-value pairs for that game. The 
possibility of hierarchical organization was included for generality and 
potential efficiency. The basic rule is that any one path through the 
data, including one list at each level, must contain all the facts for a 
single game. Also, on every such path, each attribute may occur at most 
once, unless it occurs on parallel sublists. 

The program is organized into several successive, essentially inde- 
pendent routines, each operating on the output of its predecessor and 
producing an input for the routine that follows. The linguistic routines 
include question read-in, dictionary look-up, syntactic analysis, and 
content analysis. The processing routines include the processor and the 
responder. 


WORD : WHAT PARKS DID THE RED SOX PLAY IN IN JULY? 

PART OF SPEECH: Q N D T U U Vv FY +P J 

MEANING + $23) PLACES. MONTH =JULY 
QUESTION WORD : QW 

WORD IN IDIOM: ID ID ID ID 


Fig. 13-1. Result of word look-up routine. 


13-2 The operation of the Baseball program 


To illustrate the operation of the program we shall trace the following 
question through the set of routines: “What parks did the Red Sox play 
in in July?” 

Question read-in. A question for the program is read into the 
computer from one or more punched cards. The question is formed into 
a sequential list of words. 

Dictionary look-up. Each word on the question list is looked up 
in the word dictionary and its definition copied, as shown in Fig. 13-1. 
The definition of a word in the word dictionary includes a part of speech, 
for use in determining phrase structures; a meaning, for use in analyzing 
content; an indication of whether or not the entry is a question word, 
e.g., who or what; and an indication of whether or not a word occurs as 
part of any stored idiom. Separate dictionaries are kept for words and 
idioms, an idiom being any contiguous set of words that functions as a 
unit, having a unique definition, The parts of speech might more accu- 
rately be called syntactic categories, since they are designed solely for 
use in our syntactic analysis and do not necessarily correspond with the 
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usual grammatical classes. Thus, in Fig. 13-1, parks is labeled N, mean- 
ing a noun; but July is labeled J, meaning noun-adjective alternative. 
July may be used either in the adjective position, as “The July game,” 
or in the noun position, as “in July.” Q is a particular kind of modifier 
indicated for some question words. D indicates a do-type verb, 7 an 
article, U an undefined part of speech, the associated word occurring 
only as part of an idiom, V a main verb, and P a preposition. 

The meaning of a word can take one of several forms. It may be a 
main attribute with an associated value; the meaning of July is Month 
= July, the meaning of parks is Place = , where the blank is to 
be filled in later by the content analysis. The meaning of a word may 
depend upon the immediate syntactic and semantic context in which 
the word finds itself. Such meanings are indicated as subroutines; the 
meaning of what is subroutine 23, which, when executed in the content 
analysis, will attach the value ? to the appropriate noun. Finally, some 





WORD OR IDIOM WHAT PARKS DID THE RED SOX PLAY IN IN JULY? 
PART OF SPEECH Q N D mi J Vv P P J 
MEANING $23 PLACE=_ TEAM=R. S. MONTH =JULY 
QUESTION WORD QW 


Fig. 13-2. Result of idiom look-up routine. 


words such as the, did, play, etc., have no meaning. (The function of play 
is discussed below.) 

When all the words have been looked up, any undefined words are 
printed out. If all the words are defined, the program can continue. The 
question list is scanned for possible idioms; any contiguous words that 
are indicated as parts of idioms are formed into lists for the idiom look-up 
routine. In our example parks is part of an idiom, namely, ball parks, 
but no adjacent word is also part of an idiom, and so it is ignored. How- 
ever, the string of words, the, Red, and Soz, is a potential idiom. In fact, 
the word the is part of the idiom day of the week, but at this point the 
computer must proceed as if the entire string might be an idiom. The 
idiom dictionary is consulted, and any idiom that matches this string 
or part of the string is selected. Since there is no idiom the Red Sox, the 
idiom Red Sox is accepted. Figure 13-2 shows that the two separate 
words Red and Sox have been combined into a single idiom Red Sox, with 
the meaning Team = Red Sox. 

Syntax. The syntactic analysis is based on the parts of speech, 
which are syntactic categories assigned to words for use by the syntax 
routine. There are 14 parts of speech and several ambiguity markers. 
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The syntactic analysis assumes that the question contains a single 
clause (see Sec. 13-3 on Limitations). 

First, the question is scanned for ambiguities in parts of speech 
which are resolved in some cases by looking at the adjoining words and 
in other cases by inspecting the entire question. For example, the word 
score may be either a noun or a verb; our rule is that, if there is no other 
main verb in the question, then score is a verb; otherwise it is a noun. 

Next, the syntactic routine locates and brackets the noun phrases 
[ ], and the prepositional and adverbial phrases, (_ ). The verb is left 
unbracketed. This routine is patterned after the work of Harris and his 
associates (1960) at the University of Pennsylvania. Bracketing pro- 
ceeds from the end of the question to the beginning. Noun phrases, for 
example, are bracketed in the following manner: certain parts of speech 
indicate the end of a noun phrase; within a noun phrase, a part of 
speech may indicate that the word is within the phrase, or that the word 


[WHAT PARKS] DID [THE RED SOx] PLAY IN (iN [yULY]) 





Q N D T J Vv P P J 


(IN [WHAT PARKS]) DID [THE RED SOx] PLAY (IN  [JULY]) 
SUBJECT 


ACTIVE VOICE 
NOT YES/NO 


Fig. 13-3. Result of syntactic analysis. 


starts the phrase, or that the word is not in the phrase, which means 
that the previous word started the phrase. Prepositional phrases consist 
of a preposition immediately preceding a noun phrase. The entire se- 
quence, preposition and noun phrase, is enclosed in prepositional 
brackets. In our example, as shown in Fig. 13-3, the computer brackets 
what parks, the Red Sox, and July as noun phrases and also brackets the 
prepositional phrase in July. 

Following the phrase analysis, the syntax routine determines the 
subject and object of the main verb. The content analysis often needs 
this information in generating the specification list. Sometimes, as in 
the present example, the information is irrelevant, but the program 
always obtains it anyway. But before seeking the subject and object, 
the program must take care of any dangling prepositions. Any unbrack- 
eted preposition is placed before the first noun phrase of the sentence 
that is not in a prepositional phrase, and prepositional brackets are 
added, Thus the computer improves the grammar of our question, which 
becomes Jn what parks did the Red Sox play in July? 
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Next, the syntax routine must determine whether the verb is active 
or passive. Specifically, the verb is passive if and only if the last verb 
element in the question is a main verb, V, and the preceding verb ele- 
ment is some form of the verb to be, J. In our particular example, the 
last verb element is a V, but the preceding verb element is a D, and so 
our question is in the active voice. 

When the voice has been determined, the program can use some 
simple rules to find the subject and object. For questions with active 
verbs, the free noun phrases (those not enclosed in prepositional brack- 
ets) are marked Subject and Object according to their position. The 
free noun phrase most directly preceding the last verb element in the 
question is marked Subject, and any other free noun phrase is marked 
Object. If no free noun phrase precedes the verb, the first one in the 
question is the subject and the next, if any, is the object. Since the dic- 


QUESTION 

(in. [WHAT PARKS]) DID [THE RED SOx] PLAY (IN [suLy] ) 
$23 PLACE=_ TEAM= R. S. MONTH =JULY 
PLACE=? 


SPECIFICATION LIST 





PLACE = ? 
TEAM = RED SOX 
MONTH = JULY 


Fig. 13-4. Result of content analysis. 


tionary does not contain any verbs that take two objects, a question 
will contain at most two free noun phrases. For passive verbs, the first 
free noun phrase is marked Object (since it is the object in the active 
form of the question) and all prepositional phrases with the preposition 
by have the noun phrase within them marked Subject. If there is more 
than one, the content analysis later chooses among them on the basis 
of meaning. 

Finally, the syntactic analysis checks to see whether or not any of the 
words is marked as a question word. If not, a signal is set to indicate that 
the question requires a yes/no answer. In our particular case, the word 
what was indicated as a question word, as seen in Fig. 13-2, and so the 
syntax determines that this is not a yes/no question. i 

Content analysis. The content analysis uses the dictionary mean- 
ings and the results of the syntactic analysis to set up a specification list 
for the processing program, as shown in lig, 13-4, Each noun and 
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prepositional phrase contributes one or more items to the specification 
list. Some phrases can contribute their meaning directly. The two items 
Team = Red Sox and Month = July in our example go directly onto 
the specification list. Sometimes the content analysis must determine 
the meaning of a word from its context. In the parlance of baseball, New 
York can mean Place = New York or Team = Yankees. In the dic- 
tionary of the Baseball program, the meaning of New York is sub- 
routine 1 = New York. The content analysis executes this subroutine, 
which examines the immediate context to choose a meaning. If the word 
occurs in a prepositional phrase with in or to, e.g., in New York, then 
the Place alternative is selected. 

In many noun phrases the meanings of several words must be con- 
sidered to obtain an item for the specification list. Thus Place = ? is 
obtained from the phrase what parks by combining the dictionary 
meaning of what: = ? with the dictionary meaning of parks: 
Place = . In the case of what, this is done by subroutine 23, which 
is also on the lookout for constructions like What were the parks... . 
Likewise, a number must be combined with some other word in the 
phrase, according to context. If a number follows a month name, e.g., 
May 5, it is the day. Otherwise the number must be attached as a 
modifier of the main noun in the phrase. Thus 5 teams is rendered as 
Team number of) = 5, indicating to the processor that the list of teams 
matching the other specifications must be counted and that the count 
must be 5 for a match to occur. Other combinations involve qualifiers 
that imply functions of two attributes in the data. Thus the phrase 
winning team is rendered as Teamwwinning. The value of this main 
attribute for any game is computed by a subroutine in the processor 
that picks the team associated with the larger score. 

Verbs have a variety of functions. Win, lose, beat, and their synonyms 
indicate relationships that are specified in the Baseball program by 
attaching winning and/or losing qualifiers to the attributes of the sub- 
ject and object, appropriately. Depending on the occurrence of the 
attribute Runs, the verb score may specify that a number is the value of 
the attribute Score or the modified attribute Number of Runs, the latter 
requiring the runs of all specified games to be summed. The verb éo be in 
many cases necessitates a compounding of the noun phrases which pre- 
cede and follow it, as in Who was the winning team . . . , which yields 
Team(winning) = ? from the originally separate items Team = ? and 
Teamwinning) = . Otherwise to be, as well as the verb play and its 
synonyms, has no explicit meaning. These verbs imply that a game or 
a set of games is under discussion. But a game is the normal unit of 
analysis in our data—each path is a game—and so no further specifica- 
tion is needed in our limited context, 

Verbs help to determine the associations among attributes that occur 
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on sublists in the data—i.e., Team and Score. After the content analysis 
has examined each noun phrase and the verb separately, it must examine 
the complete question to determine these associations. The form of the 
sublists on the specification list is used to indicate to the processor which 
teams and which scores must appear together (or separately) in the data 
in order for a match to occur. A team’s score appears together with that 
team on the same sublist, expressed here by parentheses (see page 260). 
Thus the question Did the Red Sox score 5? yields the specification list 
(Team = Red Sox, Score = 5), which is matched by the game on page 
260. Similarly, a team and its opponent’s score appear on different sub- 
lists. The question Was there a score of 3 against the Red Sox? yields a 
specification list with two sublists: (Team = Red Sox) (Score = 3), and 
our game on page 260 again constitutes a match. 

The content analysis derives these relationships from the question by 
an examination of the verbs and prepositions linking the attributes under 
consideration. The verbs and prepositions indicate whether the two attri- 
butes are to be put on the same or different sublists. For subject and verb 
object, the verb is the key, differentiating the two questions Did the 
Yankees score 5 runs... ? and Did the Yankees yield 6 runs... ? 
Score belongs to the “same-sublist” set of verbs, and yield belongs to the 
“different-sublist”’ set. For subject and prepositional objects, the prep- 
osition is the key, distinguishing Did the Yankees win with a score of 6? 
from Did the Yankees win against a score of 6? Again, with belongs to the 
same-sublist set of prepositions and against to the different-sublist set. 

When the specification list has been formed, it is printed in outline 
form, to provide the questioner with some intermediate feedback. 

Processor. The specification list indicates to the processor what 
part of the stored data is relevant for answering the input question and 
what information is requested about these data. The processor extracts 
the matching information from the data and produces the answer to the 
question in the form of a list structure. 

The core of the processor is a search routine that scans each path of 
a given data structure to determine whether or not all the attribute- 
value pairs on the specification list are matched by that path. A par- 
ticular pair on a specification list is considered matched on a data path 
when its attribute has been found on the path and either the data value 
is the same as the specified value or the specified value is ?, or Each, in 
which case any data value is considered a match. Matching is not always 
straightforward. Derived attributes and some modified attributes are 
functions of a number of attributes on a path and must be computed 
before the values can be matched. For example, to match Home Team = 
Red Sox, the actual home team for a particular path must be computed 
from the place and teams on that path before the specified value Red 
Sox can be matched with the computed data value, Sublists also require 
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special handling, because the entries on the sublist must be considered 
sometimes separately and sometimes as a unit in various permutations. 

When a match has been found on a given path for all the items on the 
specification list, those pairs from that path which are relevant to the 
specification list are entered on a found list. The found list produced by 
the search routine is a hierarchical list structure containing one attribute 
on each level of each path. Each path on the found list represents the 
information extracted from one or more paths of the data. The found 
list for our example might be as follows: 


Month = July 
Team = Red Sox 
Place = Boston, New York, ete. 


This found list has only one path, but the found list for the more compli- 
cated question “Where did each team play in July?” contains one path 
for each team that played in July, as shown in Fig. 13-6. On the level 
below each team, all places in which that team played in July occur on a 
list ee is the value of the attribute Place. Each path on the found list 
may thus represent a condensation i i isti P 
re Pre mane ai of the information existing on many 
Many input questions contain only one query, as in the sample ques- 
tion, Le., Place = ? These questions are answered with no further 
processing by the found list produced by one execution of the search 
routine. Other questions imply more than one search as well as addi- 
tional processing. As an example, consider the question shown in Fig 
13-5, “How many teams played in eight places in July?” This question 
is represented by the specification list labeled 4 in Fig. 13-5 and cannot 


HOW MANY TEAMS PLAYED IN 8 PLACES IN JULY? 


@ @ 
TEAMNo, of = ? TEAM = ? 
PLACENg op = 8 PLACE) o- = 8 
MONTH = JULY MONTH = JULY 
@ ® 
TEAM = EACH TEAM = EACH 
PLACE) of = ? PLACE = ? 
MONTH m JULY MONTH JULY 


Vig. U5, Derived specification tata for the indicated question. 
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MONTH = JULY 
TEAM = RED SOX 
PLACE = BOSTON, NEW YORK, - - - 


TEAM = YANKEES 
PLACE = NEW YORK, BALTIMORE, - - - 


etc. 


Fig. 13-6. Found list resulting from processing specification list 1. 


be answered directly. The question contains some implicit queries that 
must be answered first. Specifically, before the processor can count the 
number of teams, it must first determine what teams played in eight 
places in July. Again, in order to pick the teams that played in eight 
places in July, it must count the number of places each team played in 
July. Finally, in order to count the places, it must initially discover 
where each team played in July. 

The processor derives the prior specification list from the given list 
according to a list of priorities. In our example, specification list 3 is 
derived by eliminating the number modifier of the queried attribute. 
Specification list 2 is derived by replacing the definitive value of the 
number-modified. attribute by a question mark, and since only one 
question mark is allowed on a specification list, the question mark on 
Team is replaced by the query Each. Finally, specification list 1 is 
derived by the same rule that generated list 3; i.e., eliminate the number 
modifier of a queried attribute. 

When the processor has determined the prior questions that must be 
answered, it proceeds to answer them in turn. Thus, it first answers the 
question “Where did each team play in July?” by executing the search 
routine with specification list 1 as input. The result is a found list, such 
as that shown in Fig. 13-6, which includes a list of the places that each 


MONTH = JULY 
TEAM = RED SOX 
PLACEN Gg. OF = 6: BOSTON, NEW YORK, - - - 
TEAM = YANKEES 
PLACEn 9, OF = 8: NEW YORK, BALTIMORE, - -- 
etc. 


Fig. 13-7. Found list resulting from processing specification list 2. 
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MONTH = JULY 
PLACEN 9, OF = 8 
TEAM = YANKEES, TIGERS, ---- 


Fig. 13-8. Found list resulting from processing specification list 3. 


team played in July. Next, a postsearch routine answers specification 
list 2 by adding a count of each list of places on the found list. The 
results for our example are sketched in Fig. 13-7. (Note that the actual 
list of places is not erased, in case it might be needed for further process- 
ing or for responding.) When the counts have been made, specification 
list 3 can be processed. At this point the found list contains the same 
kind of attributes as the specification list; i.e., Place has been modified 
by number of. Thus the found list is logically equivalent to a data list, 
so that the search routine can be used to find paths on the found list 
that match specification list 3. In the example, we find that the Red Sox 
played in only six places, but the Yankees, the Tigers, and perhaps others 
played in eight places, as shown in Fig. 13-8. Finally, it is necessary to 
count the list of teams on the found list in order to answer specification 
list 4. The results are shown in Fig. 13-9, which is the answer to the final 
question. 

Responder. No attempt has yet been made to respond in gram- 
matical English sentences. Instead, the final found list is printed, in 
outline form. For questions requiring a yes/no answer, YES is printed 
along with the found list. If the search routine found no matching data, 
NO is printed for yes/no questions, and NO DATA for all other cases. 


13-3 Limitations 


The choice of a limited domain is undoubtedly responsible for the success 
of the Baseball program. A problem as complex as the interpretation 
of natural English is too big to be solved in a single step. A study of the 
research in automatic language translation and information retrieval 
shows that most of the significant accomplishments have been made 
with problems that have been reduced to manageable proportions. Not 
until the basic, stripped-down model has been made and has run suc- 


MONTH = JULY 


PLACE =8 


NO, OF 
TEAMyo, or = 4; YANKEES, TIGERS, --= 


Fig. 18-9, Final found list resulting from processing specification list 4, 
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cessfully is it reasonable to consider the longer and more complicated 
problems. 

A preliminary study of the question-answering problem in baseball 
made it clear that additional simplifications were needed, temporarily, 
in order to get the program started. It seemed a sufficiently big job to 
get the basic structure of a program written without solving many of 
the more complicated problems involving linguistic ambiguities. Con- 
sequently the following restrictions were imposed on input questions 
for the program described above. 


1. Logical connectives are prohibited—questions may not contain and, 
or, and not. These words imply a compounding of specification lists. 
Not also implies some elimination procedures: “Who did the Red 
Sox not play?” requires eliminating from the total list of teams those 
teams the Red Sox played. There is no major difficulty in including 
the logical connectives, but prohibiting them helped to reduce the 
problem to manageable proportions. 

2. Questions must contain only one clause—no subordinate clauses are 
allowed. If the syntax analysis is required to determine the clause 
boundaries as well as the phrase structure, a much more sophisti- 
cated program would be needed. As an interim step, the questioner 
could mark off for the computer the boundaries of his clauses, and 
the computer could deal with the subordinate clauses one at a time, 
before it deals with the main clause, using existing routines to gen- 
erate a specification list for each clause. 

3. Questions may not involve relationships such as most and highest. 
Such questions often involve many ambiguities that cannot easily 
be resolved. Determining what is meant by the question “Did the 
Red Sox win most of their games in July?” depends on a much 
larger context than the immediate question. Depending on the word 
modified by in July and on the interpretation of most, the question 
might mean any of the following: 


a. Did the Red Sox win more than half the games they played in 
July? 
b. Did the Red Sox win more games in July than in any other 


month? 
c. Did more than half the games the Red Sox won this season occur 


during July? 


The computer might answer all meaningful versions of the question 
or might even ask the questioner which meaning he intended, 
4. Questions may not involve sequential facts. To answer a question 
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such as ‘‘Did the Red Sox every play the Yankees five times ina row?” 
the program as it is now constructed would first select all games 
in which the Red Sox played the Yankees; at that point they would 
all be in a row. Somehow the program must be given the intelligence 
to select all Red Sox games and then determine whether there are 
five against the Yankees in a row. 


These restrictions are viewed as temporary expedients, and future 
work is expected to eliminate them. In most cases, the problem can 
be solved by linking two or more specification lists in order to repre- 
sent the meaning. Thus, ‘Who won the game the Red Sox lost by 
twenty runs?” requires the specification list Teamwinning) = ?, Game = 
[Teamosing) = Red Sox, Score margin = 20]; the brackets enclose a 
subordinate specification list. 

The present program, despite its restrictions, is a very useful com- 
munication device. Complex questions that do not meet the restrictions 
can usually be broken down into several simpler questions. The program 
usually rejects questions it cannot handle, in which case the questioner 
can rephrase his question. He can also check the printed specification 
list to see whether or not the computer is on the right track, in case the 
linguistic program has erred and failed to detect its own error. Finally, 
he can often judge whether or not the answer is reasonable. 

Despite its restrictions, the program can answer very many questions. 
Using the concept of the specification list, one can argue that the Base- 
ball program can answer any question whose specification list consists 
of attribute-value pairs that the program recognizes. The attributes 
may be qualified, modified, or derived, and the values may be definite 
or queries. Any combination of attribute-value pairs constitutes a speci- 
fication list. Many will be nonsense, but all can be answered. The number 
of possible specification lists is, of course, infinite, because of the numer- 
ical values. But even if all numbers are restricted to two digits, the 
program can answer millions of meaningful questions. 

The program is also capable of handling a large number of phrasings 
of the same question. Consider the following partial analysis of the 
number of possible ways of stating the question ‘‘What parks did the 
Red Sox play in in July?” This question has four major constituents: 
in what parks, did play, the Red Sox, and in July. These four elements 
may be permuted in 24 different ways. Also, there are 72 different ways 
to say in what parks, in addition to where, for a total of 73. This total 
is obtained by noting that the preposition in may occur in at least three 
locations in the question, that what has the two synonyms, which and 
which of the, and that parks has seven synonyms, park, place, places, 
ball park, ball parks, city, and cities, The product 3 X 3 X 8 = 72, In 
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the same way, there are four ways of phrasing the verb, did play, played, 
were, and took place; there are four ways of specifying the team, Red Sox, 
Red Sox Team, Boston, and Boston team; there are three ways of stating 
the month; in July, the July game, and the July ball game. The product 
of 24 * 73X44 X 3 = 84,576. Some of these forms will be ungram- 
matical, but the program can answer all of them and can also handle 
some other, more subtle ways of phrasing the same question. Of course, 
the precise number is not important, but its order of magnitude is most 
impressive. 

The present program could be markedly improved by giving it the 
facility to query the questioner, perhaps in the form of an on-line type- 
writer. Ambiguities could then be resolved by asking the questioner 
what he meant. The facility for two-way communication between man 
and computer would allow the computer to add words to its dictionary 
by asking the questioner to define new words in terms of old. Thus the 
jargon “on the road” could be defined as equivalent to the idiom “away 
from home,” which is already in the dictionary. Subtle distinctions 
among synonyms for win could be added by noting, for example, that 
edge is a synonym for beat and, in addition, means Score Margin = Less 
than 3. Man-computer conversations could also serve to train a man in 
the use of the program. After having several of his questions rejected, 
the user would soon understand what the computer could do and what 
questions were beyond its capabilities. 

Incorporating the above facilities in the Baseball program would not 
be very difficult. But an equally powerful feature that is currently 
beyond our capabilities is the facility for defining new processes for 
the computer in natural English. At present the program can do no 
more than select, count, and add. It does not now understand the con- 
cept of “standing.’”’ To answer the question ‘““Who led the League on 
July 4?”, the computer must be able to determine the won and lost, 
record of each team, compute percentages, and select the largest per- 
centage. This process appears to involve at least two or three independ- 
ent specification lists and also the new concept of percentages. 
Subroutines could be written to do this, but what is really needed is a 
program that can write its own subroutines from a description of the 
process in natural English. 

Note that we would not be satisfied with a programming language like 
FortRAN, Copot, FLrowma tic, or their relatives. These languages use 
English words to express processes, but in no sense do they interpret 
natural English. They accept only one well-defined way of stating a 
process, or only a few different ways, at best. Their grammars are precise 
and unambiguous and are quite unlike natural languages. Baseball repre- 
sents an attempt to deal with language as it is normally used by people 
unacquainted with computers. 
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13-4 Properties of the Baseball program 


Baseball differs significantly from programs for automatic language 
translation. The linguistic part of the Baseball program has as its main 
goal the understanding of the meaning of the question in order to derive 
the correct specification list. Syntax must be considered and ambiguities 
resolved in order to represent the meaning adequately. Translation 
programs have a different goal: transforming the input passage from 
one natural language to another. Meanings must be considered and 
ambiguities resolved to the extent that they affect the correctness of the 
final translation. In general, translation programs are concerned more 
with syntax and less with meaning than the Baseball program. 

Baseball differs from most other information-retrieval systems in the 
nature of its data. The term information retrieval usually refers to the 
problem of locating relevant documents. Each document has an asso- 
ciated set of index numbers describing its content. The retrieval system 
must find the appropriate index numbers for each input request and then 
search for all documents bearing those index numbers. The basic prob- 
lem in such systems is the assignment of index categories. In Baseball, 
on the other hand, the attributes of the data are very well specified. 
There is no confusion about them. However, Baseball’s derived attri- 
butes and modifiers imply a great deal more data processing than most 
document-retrieval programs. Baseball does bear a close relation with 
the Acsi-maTic system discussed by Miller, Minker, Reed, and Shindle 
(1960). 

Considerable pains were taken to keep the program general. There 
does not seem to be any obstacle to using the same computer program 
in another natural language. Different routines would be needed for 
analyzing syntax, but the meanings could be represented by the same 
specification lists. Also, most of the program can be used unchanged 
in other limited contexts, such as voting records. Answering “(How many 
states did Kennedy win in 1960?” is very similar to answering ‘How 
many games did the Red Sox win in August?” The processing program 
is indifferent to the attributes used and will handle data in any sort of 
hierarchical form. The syntax program is based entirely on parts of 
speech, which can easily be assigned to a new set of words for a new 
context. On the other hand, some of the subroutines contained in the 
dictionary meanings are certainly specific to baseball; probably each 
new context would require certain subroutines specific to it. Also, each 
context might introduce a number of modifiers and derived attributes 
that would have to be defined in terms of special subroutines for the 
processor, It is to be hoped that all such occasions for change have been 
isolated in a small area of special subroutines, so that the main routines 
can remain unaltered, However, until the experiment of switching con- 














274 Behavioral-science applications 


texts has actually been made, nothing definitive can be said about 
generality. 
part ww 


Problems 
Some computer theory 


1. What problems would be involved in transforming Baseball for another 
language, such as French or German? 

2. Plan a responder for the Baseball program that will transform the final 
found list into an acceptable English sentence. 

3. Consider applying the Baseball program to a different context. Possible con- 
texts include voting records, inventory, and census data. 

4. Discuss the applicability of the Baseball program to questions about air- 
plane or railroad schedules. 
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Inside the black box 


ire say that a computer operates by executing instructions is a 
satisfactory explanation at one level, but there remains the question 
“How does the computer execute instructions?’ An elementary answer 
will be attempted here, at the level of logical combinations of circuit 
components. The further question of how the circuit elements work will 
be avoided, since it concerns electronics (see Bartee, 1960). 

A look at the list of computer instructions in the Appendix shows that 
there are relatively few operations that must be implemented. Many of 
the operations—load, store, jump, set index, etc.—involve copying a 
word or a part of a word from one location to another. Another group— 
add, subtract, multiply, divide, index, etc.—involves addition in some 
way: subtraction is negative addition, multiplication is successive addi- 
tion, division is successive subtraction, and indexing is addition in the 
index register. A third group, the shift instructions, requires information 
to be shifted from one bit position to another in a word. A fourth group, 
the logical instructions, involves logical combinations of bits. Within the 
control unit, two other processes are important: decoding, to determine 
what instruction is to be performed, and counting, a special case of 
addition, for the program counter. Thus we have to explain copying, 
adding, shifting, combining logically, decoding, and counting. 


14-1 Components 


Computer processes can be synthesized with only three simple binary 
devices: flip-flops, and-gates, and or-gates, Mach device is made up of 
electronic resistors, condensers, and either vacuum tubes or solid-state 
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transistors, but in the present context each device will be considered 
as a single logical element. The elements are strung together by wires 
that may be treated as binary, since all computer signals are binary in 
nature. At any moment, a particular wire is at either one of two alter- 
native voltages, or levels, which may be labeled as the high level and the 
null level. Transmission of binary information occurs by means of the 
two levels. It is wrong, however, to associate a binary 1 with the high 
level and 0 with the null level, since often the correspondence is reversed. 
Nevertheless one wire, with its two possible levels, can transmit only 
1 bit at a time. One further signal can be distinguished by its temporal 
characteristic: a pulse is a brief instant of a high voltage level on an 
otherwise null line. Figure 14-1 shows a diagram of idealized pulses 
and levels. 

The main active elements in computers are the flip-flops, which are 
binary storage devices each capable of storing a single bit of information. 


High High High 


Null Null Null 
ty ty to ty 
(a) ae (6) (c) 

Fig. 14-1. Levels and pulse. (a) A low level that switches to high at time 4; (b) a 
high level that switches to low at time &; (c) a momentary pulse at t of duration 
d-to. The diagram illustrates the idealized timing assumptions that are made in the 
text concerning flip-flop switching. (a) represents the 1 output of a flip-flop, (b) the 
0 output, and (c) the 1 input. Initially the flip-flop contains a 0, and so its 0 output 
is higher. At the cessation of the pulse, both output levels switch: the flip-flop flips. 


All the registers in the processing and control sections of the computer 
are made up of flip-flops. Thus, the accumulator in our hypothetical 
36-bit binary computer consists of 36 flip-flops. The storage registers 
are not composed of flip-flops, but in this simplified logical discussion 
we may think of them as if they were. A flip-flop has two stable states. 
In one state it is said to store a 0; in the other state it is said to store a 1, 
As Fig. 14-2 shows, a flip-flop may receive any of three inputs and pro- 
duce two outputs. The inputs occur as momentary pulses. A pulse on 
the 0 input sets the flip-flop to its 0 state, while a pulse on the | input 
sets the flip-flop to its 1 state. A pulse on the complement input causes 
the flip-flop to switch states. If the state is 1, it becomes 0; if 0, it 
becomes 1. If the flip-flop is in the 0 state, the 0 output line is at the 
high level and the 1 output line is null, while if the flip-flop is in the 
1 state, the 1 output line is high and the 0 output line is null. 

The and-gates and or-gates are passive circuit elements that serve 
to control the pulses and levels, An and-gate has two inputs and one 
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output. If both inputs are at the high level, then the output is at the 
high level; otherwise it is null. Since a pulse is a momentary high level 
on a normally null line, a pulse input that meets a high-level input at an 
and-gate would get through the gate, resulting in a pulse output. If an 
input pulse encountered a low-level input, there would be no output. 
Presumably, two pulses arriving simultaneously at the inputs would 
give rise to a pulse output, but in all our uses this condition does not 
arise. An or-gate is like an and-gate in all respects except that, if ether 
of the two inputs is at the high level, 


the output is at the high level. A pulse (a) —o Pulse 
always gets through an or-gate. 

We now have all the building blocks, () = Level 
except for a source of pulses. All our 
circuits will act upon receipt of a pulse (c) energee 
or a string of pulses. For the moment, (a) 
however, we shall simply postulate the RGAE 
presence of the pulses, postponing discus- 
sion of the pulse generator until later. 

(e) Flip-flop 


14-2 Computation processes 





Copying. The simplest process to im- 
plement is copying. In Fig. 14-3, the lower Fig. 14-2... Symbols used in cir- 


row of three flip-flops contains a 3-bit cuit diagrams. The flip-flop out- 


number to be copied into the upper row 
of flip-flops. The copying occurs when 
the pulse appears from the pulse source. 
The pulse appears simultaneously at all 
the and-gates and gets through the gates 
that have a high-level input from the 
lower flip-flops. Suppose that the center 
flip-flop in the lower row contains a 0. 
Then the 0 output line of that flip-flop is 


puts are levels; the inputs are 
pulses. A 0 input pulse sets the 
flip-flop to its 0 state, in which 
the 0 output line has a high level 
where the 1 output has a low 
level. A 1 input pulse sets the 
flip-flop to its 1 state, with a high 
level on the 1 output and low 
level on the 0 output. A comple- 
ment pulse reverses the state of 
the flip-flop. 





high, and so the pulse will get through 

the and-gate connected to it and will hence set the upper flip-flop to 
0. In the same way, the pulse will get through one of the and-gates 
associated with each of the lower flip-flops, so that each of the upper 
flip-flops is set. Note that all the digits are copied at the same moment. 
This characteristic is called parallel operation, since there is a parallel 
set of circuit components for each digit position. 

Counting. A counter is a flip-flop register storing a binary number. 
Whenever a pulse occurs, the counter must increase the stored number 
by 1. The sequence of successive states of a 38-bit counter is shown in 
Table Id-l, The bit in the right-hand column of ‘Table 14-1 changes 
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Pulse source 





Fig. 14-3. Logical circuit for copying bits. 


every time the state changes; so the first requirement of the circuit is 
that the corresponding flip-flop be complemented by every pulse. The 
digit in the middle column changes only half as often. In fact, it changes 
only when the digit at its right switches from 1 to 0. Likewise, the digit 
in the left-hand column changes when the digit at its right—the middle 
column—changes from | to 0. 


Table 14-1 Successive states of a 3-bit counter{ 


States Flip-flops 
0 00 0 
1 01 
2 0 1 0 
3 01 1 
4 1 0 0 
5 101 
6 1 1.0 
7 111 


+ From state 7 the counter switches to state 0, and the cycle begins again. 


This scheme is easily realized in a logical circuit, as shown in Fig. 14-4, 
Every pulse complements the first flip-flop, as required. When the 
counter is initially at 000, the first pulse complements the first flip-flop 
but does nothing else. For consistency, we must postulate a slight time 
delay between the input to a flip-flop and the subsequent change of 
states. This time delay is at least as long as the duration of a pulse, as 
illustrated in Fig. 14-1. Thus, when the counter is initially at 000, the 
first pulse will switch the first flip-flop to 1, but the resulting high level 
on the 1 output of the first flip-flop does not allow that pulse through the 
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ulse source 


Fig. 14-4. Three-bit binary counter. 


and-gate, since the pulse has ceased before the high level occurs. The 
second pulse complements the first two flip-flops, since the high level 
from the first flip-flop allows the pulse to pass through the first and-gate. 
The second flip-flop is 0, though; so its 1 output is a null level, and the 
second pulse does not get through the second gate. The third pulse finds 
the counter in state 010. The pulse could get through the second gate 
but does not have the chance, since it cannot get past the first gate. 
Consequently, it only complements the first flip-flop. The reader can 
check that this process proceeds in the same way and easily generalizes 
to larger counters. The computer’s program counter, for example, is 
made up of a set of flip-flops in a circuit equivalent to this and is ad- 
vanced by means of a pulse at the appropriate moment. 

Shifting. Shifting is very similar to counting. Figure 14-5 shows a 
logical circuit that shifts the bit in each flip-flop to the adjacent flip-flop 
at its left, whenever a pulse occurs. The circuit also introduces zeros 
in the flip-flop made vacant by the shifting. The pulse appears simul- 
taneously at all the and-gates and at the O input of the first flip-flop. 
For each pair of flip-flops, the pulse gets through one of the and-gates 
associated with the inputs to the left flip-flop, depending on the state 





Pulse source 


Fig. 145, Logioal elroult to shift left one place, 
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of the right flip-flop. In this way, all the bits can be shifted at the same 
instant. 

Logical combination. The fourth type of computer process, logical 
combination, is straightforward with the kind of circuit components we 
are using. Figure 14-6) shows a circuit for setting the upper flip-flop to 1 
when a pulse occurs and either flip-flop contains a 1 but setting it to 0 
when a pulse occurs and neither flip-flop has a 1. If the lower flip-flop 
contains 0, then the upper flip-flop is already at the correct state, while 
the upper flip-flop must be set to 1 if the lower flip-flop is a 1. Essentially 
the same principle applies to the other two cases—setting the upper 
to 1 if and only if both are 1, and setting the upper to 0 if and only if 
both agree. 





Fig. 14-6. Logical circuits for logical combinations. (a) Upper = 1 if both are 1, 
= 0 otherwise; (b) upper = 1 if either is 1, = 0 otherwise; (c) upper = 1 if different, 
= 0 if same. 


Adding. Adding is much more complicated than the processes dis- 
cussed above, because of the problem of carrying. When adding two 
binary numbers, the sum of the digits in any particular bit position may 
result in a carry of 1 to the next position to the left. Thus the detailed 
account of an addition must include the carries explicitly. Given the 
binary integers 111 and 10110, their sum is 


xX 00111 

Y 104110 
Carry 0 110 0 
1 del evOsa 


The carry is generated from right to left. As each bit position is added, 
a carry is generated for the bit position to the left, Consequently, the 
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addition process for each bit position involves adding the digits from 
X, Y, and the carry from the right; the result is a digit for the sum and 
possibly a digit to be carried to the left. Table 14-2 shows the sum digit 
and the generated carry digit (the carry-out) for every combination of 
digits in X, Y, and the carry-in. Now, it is possible to design a circuit 
that receives the three inputs and produces the carry-out and the sum 
as specified by Table 14-2 (ignore the partial sum for the moment), but 
such a circuit has a very unfortunate property. When a set of such cir- 
cuits is built, one for each digit position, it turns out that the circuits 
cannot operate simultaneously. Each must wait for the circuit on its 
right to generate a carry-out digit, since the carry-out from one circuit 
is the next circuit’s carry-in. Thus the sum must be generated serially, 
from right to left. There is nothing peculiar about this situation—people 
add up columns of figures one column at a time from right to left. But 
it turns out that a binary computer can be more clever. 


Table 14-2 
Carry- Partial Carry- 
xX Y in sum out Sum 
0 0 0 0 0 0 
0 1 0 1 0 1 
1 0 0 1 0 1 
1 1 0 0 1 0 
0 0 1 0 0 iL 
0 1 1 1 1 0 
1 0 1 1 1 0 
1 1 1 0 1 1 


The trick is to do the addition in two steps. First a partial sum is 
formed, from only X and Y. Then the carries are generated and com- 
bined with the partial sums to produce the final sums. The carries may 
be propagated instantly so that all circuits can operate at once—in 
parallel. Table 14-2 shows that the partial sum is equivalent to the 
exclusive or; it is 0 if X and Y match, and it is 1 if X and Y differ. The 
carry-out is 1 if the partial sum is 1 and the carry-in is 1 or if the partial 
sum is 0 and Y is 1; otherwise the carry-out is 0. Finally, the sum is 
equal to the partial sum if the carry-in is 0, but the sum is the comple- 
ment of the partial sum if the carry-in is 1. 

The circuit to implement this intricate analysis is shown in Fig. 14-7, 
where the partial sum and final sum are to be in the X flip-flop. When the 
first pulse occurs, the partial sum is formed in the X flip-flop by com- 
plementing that flip-flop if Y is 1, As soon as X contains the partial 
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sum, the carry levels are correct. Thus all the carries are set up at the 
conclusion of the first pulse. At the second pulse, the carry-in is combined 
with the partial sum; ie., the partial sum is complemented if the 
carry-in is 1. 


14-3 Control processes 


Decoding. The mysterious pulses that appear in the previous dia- 
grams are sent out from the control unit. Which pulses are sent at any 


Carry-out level Carry-in level 


Second pulse 


First pulse 





Fig. 14-7. Logical circuit for a two-pulse parallel adder. 


moment depends on the operation being executed at that moment. To 
determine which operation is to be executed, the control unit must 
decode the numerical code of the operation. Suppose that we have a 
very simple computer with only four operations. The operation code 
requires 2 bits, and the control unit must be able to discriminate among 
the four codes: 00, 01, 10, 11. That is, the control unit must produce a 
pulse on one of four output lines corresponding to the four operations, 
depending on which of the 2-bit codes is encountered. Figure 14-8 shows 
the required circuit. The input pulse is switched to one of four outputs, 
depending on the state of the 2-bit operation code. 

The process of decoding is also used to select the storage register 
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referred to by the instruction. In practice, the number of bits used to 
encode the storage address can be large, in which case special switching 
problems arise. However, the logic of the situation is not altered. 
Pulse source. It is apparent by now that synchronization is critical 
in the control of a digital computer. Usually, the control unit contains 
a basic pulse source that generates pulses at a fixed rate. The control 
unit governs the computation by switching the pulses to the appropriate 


00 output 


01 output 


10 output 


11 output 


O Pulse source 





Fig. 14-8. Logical circuit for a 2-bit decoder. The three-input and-gates produce a 
high level if all their inputs are at the high level. 


units at the appropriate time. The details of the control operation are 
beyond the scope of this book. 


14-4 References 


The interested reader may find further details in a number of excellent 
books: 


Bartee, 'T. C., Jr: “Digital Computer Fundamentals,” McGraw-Hill Book 
Company, Inc,, New York, 1060, 
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Phister, Montgomery, Jr.: “Logical Design of Digital Computers,” John Wiley & 
Sons, Inc., New York, 1958. 

Richards, J.: “Arithmetic Operation in Digital Computers,” D. Van Nostrand 
Company, Inc., Princeton, N.J., 1955. 


Problems 


1. Diagram a circuit to clear a 3-bit register. When a pulse occurs, all bits are 
to be set to 0. 

2. A generalized and-gate has an arbitrary number of inputs and produces a 
high-level output if all the inputs are at the high level. Diagram an and-gate 
with four inputs and one output, using combinations of regular two-input 
and-gates. In general, how many two-input and-gates are needed to make 
an n-input and-gate? 

3. A generalized or-gate has an arbitrary number of inputs and produces a 
high-level output if any one or more of its inputs are at a high level. Diagram 
an or-gate with four inputs and one output, using combinations of two-level 
or-gates. 

4, Diagram a logical circuit for a 3-bit counter that counts backward. (When it 
reaches 000, its next state is to be 111.) 

5. Diagram a circuit similar to those of Fig. 14-6 such that the upper flip-flop is 
1 if and only if the upper and lower flip-flops match. 

6. Diagram a circuit for a 4-bit register that cycles through four states, as shown 
in Table 14-3. Each pulse causes the register to change to the next state 
in the sequence. From state 4, the register switches to state 1, beginning the 
cycle again. 


Table 14-3 
Content of 
State four flip-flops 
1 000 1 
2 00 1 0 
3 010 0 
4 10 0 0 
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Turing machines and computability 


ee diversity of digital-computer programs discussed in the pre- 
ceding chapters presents an interesting challenge to the theo- 
retician. What general statements can he make that will apply equally 
to statistical calculations, stimulus generation, pattern recognition, 
models of psychological processes, and air defense systems? About the 
only common factor is that they are all done by a digital computer. 
What does this imply? 

Tf we reduce each computation to its most general terms, we find that 
in every case the input to the computer is some string of symbols and 
the output is another string of symbols. When the product of the com- 
puter is a picture on a CRT (see Sec. 10-5), we consider the computer 
output to be the digits that control the CRT circuits. By this and similar 
arguments in other cases, all inputs and outputs can be viewed as 
symbols. The symbols can be numerals, letters of the alphabet, or any 
other specific signs that the computer can recognize. A computer, then, 
is fundamentally a device for manipulating symbols. It processes an 
input string of symbols—reorganizing, changing, adding, and eliminat- 
ing symbols—to produce an output string. A theory of computing is thus 
a theory of symbol manipulation. 

The theory of computation is an active field in mathematics today. 
While the theory has roots dating back many years, its modern impetus 
came from a paper by Turing (1936), in which he described a class of 
theoretical machines capable of manipulating symbols in a precisely 
describable way. Any computational process, and therefore any com- 
puter program, can be expressed as one of these theoretical machines, 
which have come to be ealled Turing machines, 
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Turing machines are of more than theoretical interest because of the 
generality and universality that they imply for digital computers. Not 
only can any digital computation be expressed as a Turing machine, but 
any problem that cannot be reduced to a Turing machine can legiti- 
mately be called noncomputable. Moreover, a universal Turing machine 
can be defined that can do anything that any particular machine can do, 
and any general-purpose digital computer is equivalent to such a uni- 
versal machine. Thus the contemplation of Turing machines leads to the 
conclusion that a digital computer can compute anything that can be 
computed at all. But before discussing the implications of universality 
and computability, we must first introduce the more common Turing 
machine. 


15-1 Turing machines 


A Turing machine has a finite number of internal configurations, called 
states, and operates on an infinitely long tape divided into cells. Each 
cell is capable of containing one symbol from a specified finite set of 
symbols, So, Si,..., Sy, for example, the alphabet, the numerals, ete. 
The machine deals with one cell at a time, called the scanned cell. The 
machine’s behavior is determined by its current state and by the symbol 
in the scanned cell, which is called the scanned symbol. The machine’s 
operations are limited to replacing the scanned symbol by another 
symbol, moving to an adjacent cell on the tape, and switching to another 
state. The machine starts in a certain initial state scanning a certain cell 
on the tape. On each cycle of operation, the machine reads the symbol 
in the scanned cell, replaces the scanned symbol with a specified replace- 
ment symbol, moves one cell to the right or left on the tape, and switches 
to a new state. (The new state may be identical with the old state, and 
the replacement symbol may be the same as the scanned symbol.) 

As an example, consider a machine that moves from left to right on 
a tape that contains X’s, A’s and B’s. The machine replaces each X that 
it encounters by the preceding symbol (the symbol in the next cell to 
the left at the time the X is encountered). The machine is best depicted 
by a state diagram, shown in Fig. 15-1, in which each state is shown as 
a node. State changes are indicated by arrows; accompanying each 
arrow is the scanned symbol, the replacement symbol, and the move 
indicator. Note that the direction of the move is with reference to a 
moving machine and a stationary tape, so that R means that the next 
cell is to the right of the current cell. 

In this example, the states function as a memory device. The machine 
is in state 1 when the preceding symbol was A and in state 2 when the 
preceding symbol was B. (‘This usage is reminiscent of ordinary computer 
parlance, where the state of the machine means the current content of 
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the accumulator, program counter, other active registers, and all 
memory registers.) If the machine is in state 1 and encounters an A, the 
A is unchanged (or A is replaced by A) and the machine moves right, 
staying in state 1. If the machine is in state 1 and encounters an X, the 
X is replaced by A and the machine moves right, again staying in state 
1. If B is encountered when the machine is in state 1, the B is unchanged 
but the machine switches to state 2 as it moves right. In state 2 the 
complementary situation exists: a B is unchanged and the machine stays 
in state 2, an X is changed to B and the machine stays in state 2, and an 
A is unchanged but the machine switches to state 1. In all cases the 
machine moves right to the adjacent cell. 


A;AR B;B,R 





Fig. 15-1. State diagram for X-replacing machine. The scanned symbol precedes 
the semicolon; the replacement symbol and the movement indicator follow the 
semicolon. Thus X; A, R means “If the symbol in the scanned cell is X, change it 
to A and move to the next cell at the right.’ 


The operating rules for a Turing machine can be specified precisely 
as sets of five quantities—quintuples—in the form (a,S;,S;,m,b), where 
ais the current state, S; is the scanned symbol, S; is the symbol to be put 
into the scanned cell, m is either R (move right) or L (move left), and 6 is 
the new state of the machine. There is one such quintuple for every 
combination of current state a and current symbol S;. Each quintuple 
can be thought of as a machine instruction. For example, the X-replacing 
machine can be described by the following six quintuples: 


(il, A, A, R, 1) 
(1, B, B, R, 2) 
(il, X, A, R, 1) 
(2, A, A, R, I) 
(2, B, B, k, 2) 
(2, X, B, R, 2) 


Illustrative example: counting machine. Consider a machine that 
counts consecutively. Included somewhere on the machine’s tape is a set of five 
consecutive cells containing decimal digits that comprise a five-digit decimal 
integer, At the left and at the right of the five cells are single cells containing 
the symbol ¢, as shown in Fig, 15-2, The machine, which starts somewhere to 
the left of this set of cella, is to replace the five-digit integer by the next larger 











290 Some computer theory 


Machine starts here in state 1 Machine ends here in state 4 


Fig. 15-2. Machine tape for counting machine. The d’s on the tape represent 
numerals. 


five-digit integer and is then to stop at the square following the e at the right 
of the number. The list of instructions for the machine is shown in Table 15-1, 
but the machine can more easily be understood by reference to the state diagram 
shown in Fig. 15-3. 


Table 15-1 Instructions for counting machinet 


a 


Current Scanned Replacement New 
state symbol symbol Move state 
1 Note (Self) Right 1 
1 e e Right 2 
2 Note (Self) Right 2 
2 e e Left 3 
3 0 1 Right 4 
3 1 2 Right 4 
3 2 3 Right 4 
3 3 4 Right 4 
3 4 5 Right 4 
3 5 6 Right 4 
3 6 7 Right 4 
3 7 8 Right f 
3 8 9 Right 4 
3 9 0 Left 3 
3 e e Right 4 
4 Note (Self) Right 4 
4 e e Right 5 
5 — — STOP — 


} State 1 is the initial state. For states 1, 2, and 4, the 10 quintuples in each state 
implied by “not e” are not shown explicitly. In these cases, the replacement symbol 
is equal to the scanned symbol, while the move and the new state are as indicated, 


First the machine must move right until it finds the e starting the set of digits, 
Thus, in state 1 if the current symbol is not e, the machine moves to the right 
and stays in state 1. It continues to move to the right until it finds the symbol e, 
When e is encountered in state 1, the machine again steps to the right; but now 
it enters state 2 and proceeds to search for the e at the end of the digit set, 
In state 2, if the scanned symbol is not e, the machine moves right and remains 
in state 2, thus moving past all non-e cells. When e is encountered, the machine 
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moves left to the unit digits and enters state 3. In state 3, if the current symbol 
is any one of the digits 0 to 8, the next higher digit, that is, 1 to 9, is printed 
and the machine steps right into state 4. On the other hand, if the scanned sym- 
bol is 9, it is replaced by 0 and the machine moves left to consider the next 
digit position. Fortunately, exactly the same procedure can be used in examining 
each digit position; so, following a 9, the machine remains in state 3. To provide 
for the possibility that the original number was 99999, the machine must be 
set to print 00000 but not to write over the guardian e. Consequently, in state 3, 
if the machine finds the e at the left of the digit set, the symbol remains e and 
the machine moves right into state 4. In state 4 the machine merely steps right 
until it reaches the e at the right-hand end of the digits, at which point it moves 
right once more and stops. 

Note that the machine is not specific to five-digit numbers but will work for 
any number of digits between the e’s. Note also that, if we had used different 
symbols to mark the beginning and end of the digit set, say, e at the beginning 
and f at the end, we could have eliminated state 2, for in this case state 1 could 
merely have sought the final f, ignoring the initial e. This is a simple instance 
of the general fact that the number of states can usually be decreased by increas- 
ing the number of different symbols that the machine can recognize. Conversely, 
the number of symbols can be decreased by increasing the number of states 
in a compensating way. 


A Turing machine is a model for a computer program, and the state 
diagram of the machine is a precise way of depicting such a program. 
Consequently, the phrase T’wring machine is sometimes used in the 
literature as a synonym for computer program. But a computer program 
is usually stated in larger units than individual symbols and states. The 
execution of a single instruction on a computer involves many steps of 
symbol-manipulating in the Turing-machine sense. In discussions of theo- 
retical possibilities, Turing machines and computer programs are inter- 
changeable: ‘It is possible to write a computer program that will . . .” 
has the same meaning as “It is possible to design a Turing machine that 
will. . . .” But, as a practical matter, the designer does not design 
Turing machines, except as a mathematical exercise. He programs com- 
puters instead. 


e;R e;R 0;1,R é@;R 
132,:R co 
START ae wae 2;3,R rayt 2. 5) stop 
8;9,R 
9;0,L e70,R 
Fig. 15-3, State diagram for counting machine, The symbol 6 represents all sym- 
bols except e (for example, 0, 1,2,...,9). R represents a move to the right, La 


move to the left, If the replacement symbol is the same as the current symbol, it 
is omitted from the diagram, 
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A state diagram is a convenient way of describing a process, and many 
state diagrams can be found in the psychological literature. Such dia- 
grams may have been inspired by Turing machines, but they do not as 
a rule represent Turing machines. Generally they make no reference 
to a tape or a machine. A state diagram may be complete and precise, 
but, strictly speaking, it is not a Turing machine. 

Turing-machine variations. The general Turing machine is a 
very flexible device. By adding more symbols to the set that the com- 
puter recognizes, we can reduce the number of different states needed by 
the machine, whereas by adding states we can reduce the number of 
symbols. In the illustrative example above, we have seen that one state 
can be eliminated by adding one symbol to differentiate the front and 
back of the five-digit number. The exchange is not always one for one, 
but some exchange of symbols for states is always possible. In general, it 
is easy to show that, for every Turing machine using an arbitrary, finite 
set of symbols, a similar machine can be constructed with more states 
but using only the two symbols 0 and 1. Each symbol in the original set 
must be encoded by a set of binary digits. If there are 2” original symbols, 


0;1,R 1;1,R 1;1,R 0;1,R 


Fig. 15-4. State diagram for binary form of X-replacing Turing machines. 


then n binary digits can be used to encode each symbol; the binary tape 
will have n cells for every cell on the tape of the original machine. 

Consider, for example, a binary encoding of the above X-replacing 
machine. The original symbols are A, B, and X. A convenient binary 
code will be X = 00, A = 10, B = 11. The machine will scan pairs of 
cells, from left to right, replacing 00 by the contents of the previous pair 
of cells—either 10 or 11. The machine will start on the left digit of a 
two-digit pair, in state 1. The state diagram is shown in Fig. 15-4. If 
the scanned symbol is 0, it must be the first in the X code. In state 1, 
X’s become A’s; the code for A is 10; so 1 replaces 0, and the machine 
moves one cell to the right and goes to state 3. The next symbol must be 
0; it remains 0, and the machine steps right again and returns to state |. 
In state 2, X’s become B’s; so if the scanned symbol is 0 when the ma- 
chine is in state 2, 1 replaces 0, and the machine moves right and steps 
to state 5, which replaces the second 0 of the X by 1, whereupon the 
machine returns to state 2. 

If a 1 is encountered either in state | or in state 2, the symbol is either 
A(10) or B(11). In either case the 1 is not to be changed, and the 
machine steps right to consider the next digit, in state 4. In state 4, if 
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the digit is 0, the pair was the code for A; so the machine goes to state 1, 
stepping right as it does. If the digit is 1, the pair was the code for B; so 
the machine steps right and goes to state 2. 

The binary code required the addition of three states—an inter- 
mediate switching state (4), and two intermediate printing states (3 and 
5). The three-symbol two-state machine has become a two-symbol 
five-state machine. 


15-2 Universal Turing machines 


The most remarkable of Turing’s results is his demonstration that a 
universal Turing machine exists with sufficient complexity to do any- 
thing that any other Turing machine can do. Moreover the universal 
Turing machine is not unique; a great many different universal Turing 
machines can be described. In fact, one of the current mathematical 
pastimes is to design a universal Turing machine that uses fewer symbols 
and states than any other so far invented. The apparent criterion is the 
product of symbols and states. Recently a 10-state 6-symbol machine— 
product 60—lost out to a 7-state 6-symbol machine—product 42. How- 
ever the author, Minsky (1960), admits that the tape encoding is 
“fiendishly complicated.’’ More recently, Watanabe (1961) reported an 
8-state 5-symbol machine and a 6-state 5-symbol machine.! 

The tape of a universal Turing machine contains a description of the 
particular machine to be simulated, along with the information from 
the tape of the simulated machine and some “working storage cells.”” The 
universal machine keeps on its tape a record of the current state, the 
scanned cell, and the scanned symbol of the simulated machine. It looks 
through the instructions of the simulated machine until it finds the 
instruction matching the current state and current symbol. That in- 
struction is executed, resulting in a new state and a new scanned cell. 
Different universal machines use different ways of coding the in- 
structions and the data and different devices for keeping track of the 
current conditions. 

Quite clearly, the universal Turing machine is a species of general- 
purpose digital computer. In fact, it can easily be shown that any 
general-purpose digital computer is a universal Turing machine. The 
only restriction is that the size of memory be sufficient for the problem. 
In fact, in the present discussion, the best way of demonstrating a 
universal Turing machine is by means of a computer program, as in the 
following illustrative example. 


Illustrative example: computer program for a universal Turing ma- 
chine, Figure 15-5 shows a flow diagram of a program to simulate any binary 


1Minaky reporta that bin latewt score in 28: 4 aymbolw > 7 ataton, 
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Turing machine. Since any Turing machine has an equivalent binary Turing 
machine, this computer program is a universal Turing machine. The program 
requires two large blocks of storage, one to contain the set of quintuples (in- 
structions) defining the Turing machine to be simulated, and the other repre- 


START 













Preset scanned ce!l location, current state, and final state 





Obtain scanned symbol from scanned cell 
Find instruction matching scanned symbol and state 
Which symbol is to be printed? 
it 


Store 1 
in scanned cell 


Which direction of movement? 


Right 










Store 0 
in scanned cell 





Interpret 
instruction 


Add 1 to 
scanned cell location 






Subtract 1 fron 
scanned cell location 





Change current state to next state 
Does current state match final state? 





STOP 


Fig. 15-5. Flow diagram of universal Turing machine. 


senting the tape of that machine. In considering the program, it is easiest to 
suppose that both these blocks are in core storage and that there is adequate 
core storage to contain all the necessary registers to simulate the tape. Moreover 
we shall store only one symbol in each register in the block simulating the tape, 
We shall also.use one register per instruction in storing the machine description, 
Each instruction register is divided into five parts, containing the codes for 
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the initial state, the current symbol, the replacement symbol, the move, and the 
next state of the machine. The current symbol, the replacement symbol, and 
the move each requires only 1 bit. The remaining bits can be divided equally 
between the codes for the two state symbols. 

The flow diagram in Fig. 15-5 is remarkably simple and straightforward, as 
digital computer programs go. The only part that requires more than a few 
machine instructions is the process of finding the instruction that matches the 
current state and the scanned symbol. This is a kind of table look-up operation 
in which the parts of each stored instruction containing the current state and 
scanned symbol must be compared with the actual state and scanned symbol 
on that cycle, until a matching instruction is found. 

Note that there is no limitation on complexity as such, but only on size. 
If the memory blocks for the instructions and the tape were infinitely large, 
with the registers in the instruction block each being infinitely long, to accom- 
modate state codes of any length, then the machine would be completely uni- 
versal. The program as it stands can always be expanded to accommodate 
larger and larger blocks of instructions and data, and can even use magnetic 
tapes, without altering the logic of the flow diagram. 


We have shown that any digital computer can behave like a universal 
Turing machine. We could readily show the converse, that, for any 
digital computer, there is a Turing machine that behaves in the same 
way. To do so, we would merely have to show that each of the logical 
building blocks in the computer—the flip-flops, and-gates, and or-gates 
—can be represented in Turing-machine notation. Then the complete 
configuration of elements comprising the logical design of a particular 
computer could readily be written as a Turing machine. The actual 
demonstration is rather extensive, and will be left for the reader. Here 
we shall merely assert that any general-purpose digital computer is 
equivalent to a universal Turing machine. 

It might seem that a digital computer must be more powerful than a 
universal Turing machine. A program based on the flow diagram of 
Fig. 15-5 would certainly not tax a digital computer. To add two multi- 
digit numbers in a Turing machine takes many processing steps and 
many states, whereas a computer can add two numbers in one operation. 
But that one operation, when reduced to its logical elements, is in fact a 
collection of many operations. The computer is an organization of ele- 
ments that is easier to use than a Turing machine is to design and in 
that sense is more powerful. But the computer can do no more than a 
universal Turing machine. Both can compute anything that any Turing 
machine can compute, 

The only distinction between a real computer and a Turing machine 
is the latter’s theoretically infinite storage capacity. This distinction is 
not so critical and fundamental as might be supposed, In any particular 
computation, a Turing machine does not use an infinite length of tape. 
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At the end of a computation a machine will have used a finite amount 
of tape. Otherwise the computation would not have stopped. It takes 
forever to use an infinite tape. What is meant by infinite is that there is 
no limit on the amount of tape used. If more tape is needed, more is 
always available. The capacity of any real machine can always be 
extended without limit by human intervention. The machine can signal 
an operator that it has run out of tape, and a new tape can be installed. 
Likewise, the machine can call for an old tape to be reinstated. With 
such a signaling system and with a human operator, any general-purpose 
digital computer becomes an unlimited universal Turing machine. 

If all general-purpose digital computers are universal, then each can 
simulate the behavior of the others. What can be done on one computer 
can also be done, more or less efficiently, on another. An immediate 
practical application arises in computer design. The logical functioning 
of a new computer in the design stage can be simulated in detail on an 
existing computer. Any bugs in the design can be caught before the new 
computer is actually built. Also, programs for the new machine can be 
written and tested before it is built, so that it will be ready to go to work 
as soon as the last wire is soldered. 

A program that makes a computer behave like another computer is an 
interpreter (see Chap. 5). The program diagrammed in Fig. 15-5 is an 
interpreter for a Turing machine. It interprets codes for Turing-machine 
instructions and executes the interpreted instructions. Any digital com- 
puter can be supplied with an interpreter for the instructions of any 
other digital computer. The interpreter will not be very different in design 
from that in Fig. 15-5. Its cycle of behavior must include finding the next 
instruction, decoding it, and executing it. Because an interpreter requires 
many steps to execute one instruction, it is much slower than the 
machine being simulated. But if the simulated machine is nonexistent 
or unavailable, an interpreter may be economically justified. 


15-3 Computability 


The mathematical interest in Turing machines arises from the problem 
of stating precisely what is meant by a computable function. Turing 
defined computability in terms of his machine; a computable function 
is one that can be computed by a Turing machine. Other mathematicians 
have given nearly equivalent definitions. For a complete mathematical 
discussion, see Rogers (1959) or Davis (1960). (Nagel and Newman, 
1959, give a readable account of Gédel’s theorem, which is one aspect of 
computability.) 

At first glance Turing’s definition of computability might seem 
restrictive, but it turns out that any finite symbol-manipulation process 
can be represented as a Turing machine. A computation is a manipue 
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lation of an input string of symbols to obtain an output string of symbols. 
The output can be said to be a function of the input. This function is 
computable if a finite set of rules can be stated for manipulating the 
input symbols so that, after a finite amount of processing, the output is 
produced. Any finite set of rules can be represented by the states and 
interconnections of a Turing machine. Recall that there is no limit on the 
number of states in the machine, except that the number must be finite. 
Whatever intricate interdependence the process may involve, a combi- 
nation of states and tape storage will suffice to keep track of it. 

Finding a noncomputable function of practical interest is difficult. 
Most noncomputable functions are subtle decision problems. Suppose, 
for example, that we are confronted with a particular Turing machine 
in a certain initial state, and consider the problem of determining 
whether the machine will eventually reach the final stopping state. It 
might not, because of the peculiar nature of some input. For some Turing 
machines the problem is easy; for others it is difficult. As a general 
problem it is noncomputable. Mathematicians have proved that it is not 
possible to construct a Turing machine that will accept as input the 
specifications of any arbitrary Turing machine and will guarantee to 
compute whether or not the specified machine is sure to reach its final 
state, no matter what its input. This result is philosophically satisfying, 
for it says, in effect, that the problem of determining whether or not a 
function is computable is itself noncomputable. But the result is of little 
practical importance. In any particular case, one can almost always 
determine whether a function is computable; and in almost all practical 
cases, the function will be computable. 

When a problem is very complex and its solution is elusive, there is a 
great temptation to conclude that the problem is noncomputable. Such 
a conclusion is seldom justified. Problems that seem at first glance to be 
noncomputable are more likely to be poorly specified. Pattern recog- 
nition is a case in point. The problem of recognizing the words on a 
printed page or a sheet of typescript is certainly computable. In this 
case we are able to specify the criterion sufficiently so that a mathe- 
matical problem exists—the problem is “well formed.” But in recognizing 
handwriting the only reasonable criterion is a comparison with the 
judgment of human observers, and this is not an acceptable specification 
for a mathematical problem. We are in no position to determine com- 
putability, because our problem is not well formed. This situation is not 
atypical. In general the issue is usually the precise specification of the 
problem—once this has been done, the solution is usually easy. 

Even if a function is noncomputable, all hope is not lost. A non- 
computable function is one for which no computer program can guaran- 
tee a solution, But there may still be a computer program that will some- 
times solve the problem, We noted above that no computer program can 
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guarantee to determine whether or not any specified Turing machine 
will be sure to stop, but a computer program might be devised that would 
work for some of the specified Turing machines. Such a program might 
in fact use the same kind of reasoning processes that human mathe- 
maticians use in studying particular Turing machines. The computer 
might have many potentially useful procedures or heuristics, like the 
problem-solvers discussed in Sec. 11-4. Such a program might be able to 
make considerable headway on noncomputable problems. 

Heuristic programs are also useful for computable problems. Chess, 
for example, is a computable problem, but the program that guarantees 
to beat anyone must follow through each possible move to its ultimate 
conclusion. Billions of years would go by between moves. On the other 
hand, a very good chess player can be programmed to choose a move in a 
few minutes, using the heuristics that good human players use. Such a 
machine is not infallible, but it may be very good, and it is certainly 
attainable, whereas the dull plodder that would eventually win is 
certainly impractical. 


15-4 Efficiency 


Proving that a particular function is computable is merely proving a 
possibility, as we have seen in the chess example. If a function is com- 
putable, then a Turing machine can be designed to compute the function 
and, because of the equivalence between digital computers and uni- 
versal Turing machines, any digital computer can also be programmed 
to compute the function. The theory has nothing to say about the 
efficiency of the computation. 

Efficiency is a vital practical concern. This is what computers are all 
about. Computers are not much help with calculations that can be done 
speedily and cheaply by hand, but calculations that would be ridicu- 
lously inefficient to do by hand can often be done with dispatch on a 
digital computer. Some problems are most efficiently done on a small 
computer, others require a large, high-speed machine, and still others 
cannot be done efficiently on any existing machine. Which computer to 
use or whether to use a computer at all is a decision that depends on the 
time and money involved in each course of action. 

Differences among machines are solely differences in efficiency. In 
practice these differences can be very large; nevertheless it is important 
to realize that efficiency is the only consideration. To say that a problem 
is beyond the capacity of a machine is to say that the problem must be 
split into segments to fit into it and that human intervention is required 
to extend its storage capacity or else that it is so slow and the problem 
so long that too many hours of computing would be required, If differ- 
ences among digital computers are measured solely in terms of efficiency, 
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it must follow that digital machines built for special purposes—e.g., the 
perceptron, speech-recognizing machines—are built only for the sake of 
efficiency. The special machines will presumably be much more efficient 
at doing the specific operations required. But any such machine can 
always be simulated on a general-purpose digital computer, and in the 
design and development stages simulation may be preferable, because 
of the relative ease of making changes. An instruction in a program can 
be changed more easily than a machine can be rewired. 

In the end, then, the theory has been encouraging about the scope of 
computers but not very helpful about practical matters. The computer 
can do almost anything we can conceive of asking, but we must count 
the costs before making the request. It has been our thesis in this book 
that the costs are not great and that scientists will be willing and eager 
to use computers when they see what power and versatility can be ob- 
tained. Computers offer the most economical means of doing a wide 
variety of jobs and allow scientists to do research that could be done in 
no other way. If the cost of not doing research could be determined, the 
contribution of computers to the efficiency of science would be enormous. 
Digital computers, used with intelligence and imagination, are indis- 
pensable in the progress of science. 


Problems 


1. No provision was made for stopping the X-replacing machine described in 
the text. By adding the symbol Y to the symbol set, we can specify that the 
machine shall stop when Y is encountered. Implement this specification by 
making the necessary corrections to the state diagram in Fig. 15-1. 

2. Make the necessary corrections to the state diagram of Fig. 15-4 so that the 
machine will stop when the pair 01 is encountered. 

3. If a different binary encoding is chosen for the X-replacing machine, an 
additional state is needed. Make the state diagram for the machine if the 
encoding is X = 00, A = 01, B = 10. 

4. Specify a binary encoding for the counting machine in the illustrative exam- 
ple, and draw the state diagram for the equivalent binary Turing machine. 

5. Write a coded program for a universal Turing machine from the flow dia- 
gram in Fig. 15-5. Use the code of any real computer or the hypothetical 
computer in this book. 

6. The MIT TX-0 computer, described by Clark (1957), has only three opera- 
tions that refer to storage: a computation instruction, ADD X; a transmission 
instruction, STORE IN X; and a conditional jump instruction, JUMP IF 
MINUS TO X. The TX-0 also has a small collection of operations that do 
not affect storage, including: CLEAR ACCUMULATOR, COMPLEMENT 
ACCUMULATOR, ROTATE ACCUMULATOR LEFT 1, READ WORD 
TO ACCUMULATOR, WRITE WORD FROM ACCUMULATOR, and 
HALT, The TX-0 uses onescomplement arithmetic, and the first bit in a 
register is the sign bit, JUMP LE’ MINUS is equivalent to JUMP IF FIRST 
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BIT = 1. Thus a comparison can be made by this instruction combined with 
ROTATE LEFT. Show that the TX-0 is a universal Turing machine by 
showing how the TX-0 can be coded for the program of Fig. 15-5. 

Write a flow diagram for an interpretive program that permits a certain 
computer to simulate the operation of another computer. For example: 


a. Simulate the behavior of the TX-0 on our hypothetical computer. 
b. Simulate the behavior of our hypothetical computer on an existing com- 
puter. 


Make a state diagram for the following Turing machine: The tape is blank 
except for a sequence of A’s and B’s between two e’s, of the form 


eABBBAABA-::: Be 


The machine starts by examining the leftmost A or B. If the symbol is A, 
the machine is to add the two symbols A A to the right-hand end of the 
string of A’s and B’s after moving the right-hand e two cells to the right. 
If the symbol is B, the machine is to add the string of four symbols B B A B 
to the right-hand end of the string, moving the right-hand e four places to 
the right. Then the machine is to move the left-hand e three places to the 
right, erasing the three leftmost symbols in the A B string. The entire process 
is repeated iteratively until everything is erased, which may never happen. 

The interest in this simple process lies in the computability of its decision 
function, the function that tells for a given initial string whether the machine 
will ever stop. This decision function may or may not be computable. No one 
knows, at the present writing. Some similar processes only slightly more com- 
plicated are known to have noncomputable decision functions. Thus the issue 
of computability can arise in problems that appear very simple. 
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Instructions for a hypothetical computer 


The following list of instructions for a binary, single-address computer can be 
used to program the examples in the text. In the description, Y stands for the 
address of a storage register, and N stands for a positive integer (including zero). 
Unless otherwise specified, the program counter is advanced in the normal 
manner when each instruction is executed; i.e., 1 is added to the contents of 
the program register. Unless otherwise specified, each instruction that includes 
Y or N can be indexed. An index register is designated by a letter: A, B,C, D,... 
(assume as many different index registers as necessary). An indexed instruction 
is indicated by appending to the base address a colon followed by the designa- 
tion of an index register, that is, Y:A. The effective address is the sum of the 
base address and the content of the index register. The descriptions are given 
for the instructions with no indexing. For an index, read Y + I for Y and N + I 
for N in the descriptions, where I is the content of the designated index 
register. 


Data-transmission operations 


LOAD FROM Y 
The content of register Y is placed in the accumulator, erasing the previous 
content of the accumulator. The content of Y is not altered. 

STORE IN Y 
The content of the accumulator is placed in register Y, erasing the previous 
content of Y. The content of the accumulator is not altered. 

STORE ADDRESS IN Y 
The digits in register Y that encode the address purt of a computer instruction 
are replaced by the corresponding digits in the accumulator, The other digits 
in register Y are not altered, The content of the accumulator is not altered, 
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STORE OPERATION IN Y 
The digits in register Y that encode the operation part of a computer instruc- 
tion are replaced by the corresponding digits in the accumulator. The other 
digits in register Y are not altered. The content of the accumulator is not 
altered. 

STORE INDEX IN Y 
The digits in register Y that specify the index register associated with a com- 
puter instruction are replaced by the corresponding digits in the accumulator. 
The other digits in register Y are not altered. The content of the accumulator 
is not altered. 

STORE REMAINDER IN Y 
The content of the remainder register is stored in register Y, erasing the pre- 
vious content of Y. The content of the remainder register is not altered. 
(The remainder register holds the remainder after a division and the least 
significant part of the product after a multiplication.) 


Arithmetic operations 


The following operations are for fixed-point arithmetic and may be used with 
integers or fractions. If the result of any operation is 0, it is always +-0. Details 
about overflow are omitted. (See Sec. 6-5. Programs in Chaps. 3 and 4 use only 
fixed-point arithmetic, and it is assumed that all numbers are small enough so 
that overflow does not occur.) 


ADD CONTENT OF Y 
The content of Y is added to the content of the accumulator, the result being 
left in the accumulator. The content of Y is not altered. 

SUBTRACT CONTENT OF Y 
The content of Y is subtracted from the content of the accumulator, the result 
being left in the accumulator. The content of Y is not altered. 

MULTIPLY BY CONTENT OF Y 
The content of the accumulator is multiplied by the content of Y. The most 
significant part of the result is left in the accumulator. The least significant 
part is put in the remainder register. The content of Y is not altered. 

DIVIDE BY CONTENT OF Y 
The content of the accumulator is divided by the content of Y; the quotient 
is put in the accumulator. The remainder is put in the remainder register. The 
content of Y is not altered. If the content of Y is zero, the computer will stop, 


The following operations are for floating-point calculations. They expect 
floating-point operands and produce floating-point results. Their descriptions 
are identical with their fixed point counterparts, except for details that are 
omitted. 


FLOATING ADD CONTENT OF Y 
FLOATING SUBTRACT CONTENT OF Y 
FLOATING: MULTIPLY BY CONTENT OF Y 
FLOATING DIVIDE BY CONTENT OF Y 





Instructions for a hypothetical computer 303 


Control operations 


Note that the number zero is always positive. In both fixed and floating-point 
format, zero is represented by the digit 0 in every digit position. 


JUMP TO Y 
The program counter is set to Y so that the computer gets its next instruction 
from register Y. 

JUMP IF MINUS TO Y 
If the accumulator contains a negative number, the program counter is set 
to Y, so that the computer gets its next instruction from register Y. Otherwise, 
the program counter is advanced in the normal manner, so that the computer 
gets its next instruction from the next consecutive register. 

JUMP IF PLUS TO Y 
If the accumulator contains a positive number, the program counter is set 
to Y. Otherwise, the program counter is advanced normally. 

JUMP IF ZERO TO Y 
If every bit position in the accumulator contains a zero, the program counter 
is set to Y. Otherwise the program counter is advanced normally. 

SKIP IF 1 IN BIT N 
If the digit in bit position N in the accumulator is 1, then the computer skips 
the next instruction in sequence, going to the following instruction. If the 
digit in bit position N in the accumulator is 0, the computer takes the next 
instruction in sequence. (The leftmost digit is position 1; the rightmost is 36.) 

SKIP IF ON SWITCH N 
If the console switch numbered N is on, the computer skips the next instruc- 
tion in sequence, going to the following instruction. If switch N is off, the 
computer executes the next instruction in normal sequence. This instruction 
cannot be indexed. 

HALT 
The computer stops, turning on the STOP light on the console. If restarted 
manually, the computer will execute the next instruction in normal sequence. 

NO OPERATION 
This instruction is vacuous. It has no effect other than to advance the program 
counter in the normal manner. 


Index operations 


The following instructions all refer to index register A. A completely parallel 
set exists for each index register in the computer. Assume as many index registers 
as necessary for programming a problem. These instructions cannot themselves 
be indexed. Y is always the base address. That is, the form Y:B is illegal. 


LOAD A FROM Y 
The content of storage register Y replaces the content of index register A; 
the content of Y ia not altered, 
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STORE A IN Y 
The content of index register A replaces the content of storage register Y; 
the content of A is not altered. 

INDEX A FROM Y 
The content of storage register Y is added to the content of A, and the sum 
is left in A. The content of Y is not altered. 


JUMP IF A MINUS TO Y : 
If index register A contains a negative number, the program counter is set to Y. 


Otherwise the program counter is advanced in the normal manner. 

JUMP IF A PLUS TO Y 
If index register A contains a positive number, the program counter is set to Y4 
Otherwise the program counter is advanced in the normal manner. 

SET A AND JUMP TO Y 
The address of this instruction (the content of the program counter) replaces 
the content of index register A. The program counter is set to Y. 


Logical operations 


MASK FROM Y 
The content of the accumulator is compared, bit by bit, with the content 


of register Y. Wherever the bit in the accumulator is 0, it remains 0. Wherever 
the bit in the accumulator is 1, it is replaced by the value of the corresponding 
bit in register Y (logical and). The content of register Y is not altered. 
¥ . 

sere ae of the accumulator is compared, bit by bit, with the content of 
register Y. Wherever the bit in the accumulator is 0, the corresponding aa 
in register Y is unchanged. Wherever the bit in the accumulator is 2 ee 
corresponding bit in register Y is made a 1 (logical or). The content of the 
accumulator is not altered. 


MATCH WITH Y a 
The content of the accumulator is compared, bit by bit, with the content 


of register Y. Wherever the 2 bits are the same, the bit in the accumulator is 
set to 0; wherever the bits are different, the bit in the accumulator is set to 1 
(logical exclusive or). The content of register Y is not altered. 


COMPLEMENT 
Each bit in the accumulator is reversed. Each 0 becomes a 1, and each 1 


becomes a 0. 


Shift operations 


These instructions refer to individual digit positions in the accumulator. Posi- 
tions are numbered from 1 to 36, from left to right. 


SHIFT LEFT N me is 
The digits in the accumulator are shifted left N bit positions. The digit in 


bit position 36 is shifted to bit position 36 — N, ete. Digits shifted past 
bit position 1 are lost. Bit positions made empty by the shift are filled with Ox, 
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SHIFT RIGHT N 
The digits in the accumulator are shifted right N bit positions. The digit 
in bit position 1 is placed in bit position N + 1, etc. Digits shifted past bit 
position 36 are lost. Bit positions made empty by the shift are filled with Os. 
ROTATE LEFT N 
The digits in the accumulator are shifted left N bit positions, as in SHIFT 
LEFT N. However, the digits shifted left past bit position 1 are brought 
around and entered in the bit positions at the right end of the accumulator 
emptied by the shift, in such a manner that ROTATE LEFT 36 does not 
alter the content of the accumulator. 


In-out operations 


The following partial list of in-out instructions assumes that there is only one 
input device and one output device. The in-out medium may be considered 
to be punched paper tape or magnetic tape. The magnetic tape may be con- 
sidered to contain images of punched cards, one row on the tape corresponding 
with one column on the card. The contents of the tape can be printed off-line 
if it contains alphanumeric codes. 


WRITE CHARACTER 
The rightmost 6 bits in the accumulator are written as a row on the tape. 
Any code may be used. If the standard alphanumeric code is used, the content 
of the tape can be printed off-line. The content of the accumulator is not 
altered. 

WRITE WORD 
The sequence of instructions ROTATE LEFT 30 and WRITE CHARACTER 
is executed six times, resulting in six rows on the tape, one for each 6-bit 
segment in the accumulator, starting with the leftmost 6 bits. The content 
of the accumulator is not altered. 

READ CHARACTER 
A row of 6 bits is read from the tape and placed in the rightmost six bit posi- 
tions in the accumulator (bits 31 to 36). The other bit positions in the accu- 
mulator are not altered. 

READ WORD 
The sequence of instructions ROTATE LEFT 6 and READ CHARACTER 
is executed six times, resulting in six rows on the tape being read into the 
accumulator, the first row into bits 1 to 6, etc. The previous content of the 
accumulator is erased. 

PRINT CHARACTER 
The character corresponding to the rightmost 6 bits in the accumulator is 
printed on an on-line printing device. The content of the accumulator is not 
altered. An arbitrary 6-bit code is used, in which the codes for the numerals 
0 to 9 are the corresponding binary numbers 000000 to 001001. (Some of the 
codes are for special characters; others are for printing control, e.g., starting 
a new line, skipping a line, ete, Assume values for any needed operations and 
symbols.) 
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Bootstrap procedure 


To provide a sample of how something gets into the computer initially, we ar- 
bitrarily define a button on the console labeled LOAD TAPE. When this is 
pushed, the first eight words on the input tape are read into the computer and 
stored in the first eight registers (registers 0 to 7), respectively. Then the com- 
puter takes its next instruction from register 2. 
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answers 


Answers are provided for a few of the problems in the text. Many equivalent answers 
are possible for some of the problems, including all problems for which the answer is 
a computer program. An answer that is listed here for such a problem is one possi- 
bility, but it is not necessarily the best. 


Chapter 3 

3-2c 
100 LOAD FROM 121 
101 SUBTRACT CONTENT OF 122 
102 JUMP IF MINUS TO 105 
103 STORE IN 120 
104 JUMP TO (Next) 
105 LOAD FROM 122 
106 SUBTRACT CONTENT OF 121 
107 JUMP TO 103 
120 (ANSWER) 
121 X 
122 Y 


3-3 Table 3-6 
Eliminate 131, 132, 133. Change 154 to ADD CONTENT OF 240. 
3-3 Table 3-9 
Eliminate the unconditional jump within the loop: 
(120 to 122 unchanged) 


128 JUMP 'TO 131 
317 
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124 
125 
126 
127 
128 
129 
130 
131 
132 
133 


ADD CONTENT OF 155 
STORE IN 127 
LOAD FROM 250 
ADD CONTENT OF — 
STORE IN 250 
LOAD FROM 151 
SUBTRACT CONTENT OF 153 
STORE IN 151 
JUMP IF PLUS TO 124 
JUMP TO 500 


3-3 Tables 3-10 and 3-11 


Use the same principle as in Table 3-9. 


3-4 


Use a table look-up procedure to find the proper jump instruction. 


3-5 


3-66 


100 
101 
102 
103 
104 
105 
106 
107 
108 
120 
121 
122 


116 
120 
121 
122 
123 
124 
125 


3-10 


100 LOAD A FROM 121 
101 LOAD FROM 200:A 
(Eliminate 102, 103.) 


LOAD A FROM 122 
LOAD FROM 200:A 
JUMP IF PLUS TO 106 
LOAD FROM 120 
SUBTRACT CONTENT OF 200:A 
STORE IN 200:A 
INDEX A FROM 121 
JUMP IF A PLUS TO 101 
(Next program) 

0 

—1 

+19 

(Next program) 

—300 

0 

+1 

—4 

-1 


See illustrative example of integer print-out in Chap, 4, 
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Chapter 5 
5-1b 


po 11 I 
pO 11 J 
IF (A(I) — AGVJ)) 
10 TEMP = A(I) 
A(I) = AW) 
A(J) = TEMP 
11 CONTINUE 


=1,N-1 
=I+1,N 
)) 10, 14,11 


5-2 
Replace the statements in Table 5-4, starting with DO 20, by: 


WRITE OUTPUT TAPE 3, 101, N, FMEAN, SIGMA 
DO 30 I=1,N 

20 TEMP = (X(I) — FMEAN)/SIGMA 

30 WRITE OUTPUT TAPE 3, 102, X(I), TEMP 
STOP 


Format 101 is the same as in Table 5-4, except that ) is added at the end of continua- 
tion 2 and continuation 3 is replaced by 102 FORMAT (2F 10.5). 


5-4 


Note that constructions of the form R(100 — I, 100 — J) are illegal in Fortran. 
Some circumlocution is necessary, such as: 


IX = 100 —I 
JX = 100 —J 
R(X, JX) = ++: 


5-5 
Fach parenthetical expression requires a sublist. Thus ((S/F) — (A*B))/(C*D) is: 


Main list: Sublist 1, 
/ ‘ 

Sublist 2. 

Sublist 1: Sublist 3, 
Sublist 4. 

Sublist 2: Cc ; 


* 
’ 


Sublist 3: 


Subliat 4; 


al) 
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5-59 


Main list: 


Description list 1: 


Sublist 1: 


Description list 2: 


Sublist 3: 
Description list 4: 


Description list 1, 
Sublist 1, 

Sublist 2. 
PLACE, 
BOSTON; 
MONTH, 
APRIL; 

DAY, 

29. 

Description list 2 
Sublist 3, 

Sublist 4, } 


Sublist 11. 
TEAM, 

RED SOX; 
RUNS, 

5; 

HITS, 

7 

ERRORS, 

1. 

Description list 4 
PLAYER, 
SCHILLING; 
POSITION, 
SECOND BASE; 
TIMES AT BAT, 
4; 

HITS, 

2; 

RUNS, 

1. 


Sublist 2 has description lists and sublists for the opposing team. The sublists and 
description lists for each player have the same form as that shown here. Other attri- 


butes may be added as needed. 


Chapter 6 


6-7 


There are two sets: 001, 010, 100, and 111; 


and 110, 101, 011, and 000, 





Chapter 7 


7-la 


7-2 


T-4 


LEFTJ 


SHIFT 


LMASK 
BLANK 
TEMP 


LOOP 


INX 


COUNT 


LDATA 
ZERO 
MONE 
ONE 
NINTEN 
MASK1 
MASK2 


ONES 


STORE IN 

LOAD FROM 
MASK FROM 
MATCH WITH 
JUMP IF ZERO TO 
LOAD FROM 
JUMP TO 

LOAD FROM 
ROTATE LEFT 
JUMP TO 
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TEMP 
LMASK 
TEMP 
BLANK 
SHIFT 
TEMP 
1:A 
TEMP 
6 
LEFTJ 


(Bits 1-6 ones, 7-36 zeros) 
(Bits 1-6 code for blank, 7-36 zeros) 


LOAD B FROM 
LOAD A FROM 
LOAD FROM 

MASK FROM 
MATCH WITH 
JUMP IF ZERO TO 
INDEX A FROM 
JUMP IF A PLUS TO 
STORE B IN 

JUMP TO 

STORE A IN 

LOAD FROM 

ADD 

STORE ADDRESS IN 
INDEX B FROM 
JUMP TO 

0 

-1 

+1 

+19 


(Bits 3, 7, 10, 12, 13 one; 


ZERO 
NINTEN 
DATA:A 
MASK1 
MASK2 
COUNT 
MONE 
LOOP 
NFOUND 
(Next program) 
TEMP 
LDATA 
TEMP 
FOUND:B 
ONE 

INX 
DATA 


others zero) 


(Bits 3, 7, 10 one; others zero) 


STORE B IN 
STORE C IN 
LOAD B FROM 
LOAD C FROM 
COMPLEMENT 


SAVEB 
SAVEC 
THRFVE 
ZERO 
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7-10 


Answers 


LOOP 


ONE 
ZERO 
THRFVE 
MONE 
TEMP 
SAVEB 
SAVEC 


INTEGR 


LOOP 


ZERO 
MONE 
FIVE 
DIGIT 
SAVEB 
TEMP 
INT 
TEMP2 


Chapter 8 


8-1 


sin? « + cos’? = 1. 


SKIP IF 1 IN BIT 
INDEX C FROM 
INDEX B FROM 
JUMP IF B PLUS TO 
STORE C IN 
LOAD FROM 
LOAD B FROM 
LOAD C FROM 
JUMP TO 

+1 

0 

+35 

—1 


STORE B IN 
STORE IN 

LOAD FROM 
STORE IN 

LOAD B FROM 
LOAD FROM 
ROTATE LEFT 
STORE IN 

MASK FROM 
STORE IN 

LOAD FROM 
SHIFT LEFT 

ADD CONTENT OF 
ADD CONTENT OF 
ADD CONTENT OF 
STORE IN 

INDEX B FROM 
JUMP IF B PLUS TO 
LOAD B FROM 
JUMP TO 

0 

-1 

+5 


1:B 
ONE 
MONE 
LOOP 
TEMP 
TEMP 
SAVEB 
SAVEC 
1:A 


SAVEB 
TEMP 
ZERO 
INT 
FIVE 
TEMP 
6 
TEMP 
DIGIT 
TEMP2 
INT 

3 

INT 
INT 
TEMP2 
INT 
MONE 
LOOP 
SAVEB 
1:A 


(Bits 31-36 one; others zero) 
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Chapter 9 


9-1 


Locate the input symbol in the stored list of five symbols, and interchange it 
with the fifth symbol; then select one of the first four symbols at random. Or 
select one of the five symbols at random, and if the input symbol is selected, 
try again, repeating until the selected symbol differs from the input symbol. 


Step 1. Preset: nj = 25, for 7 = 1, 2, 3, 4; COUNT = 50. 

Step 2. Select one at random from four unequally likely alternatives, with the 
probability of stimulus 7 being ni/(m1 + ne + nz + m4). 

Step 3. Subtract 1 from the n corresponding to the selected stimulus. Subtract 
1 from COUNT. Repeat steps 2 and 3 if COUNT is greater than 0. 


9-5 c«=ay=ratVil—? db. 
Chapter 14 
14-2 Two and-gates each receive two of the four inputs. The outputs from these 


14-3 


14-4 


14-5 


14-6 


two gates are the inputs to a third and-gate, the output of which is the output 
of the generalized gate. 


Same as 14-2, with or-gates rather than and-gates. 


Same as Fig. 14-4, but with the outputs of the flip-flops coming from the zero 
side. 


Same as Fig. 14-6c, but with the output of the lower flip-flop coming from the 
zero side. 


Four flip-flops in a row; the 1 output of each flip-flop meets the pulse at an 
and-gate, the output of which goes both to the 0 input of the same flip-flop 
and the 1 input of the flip-flop to the left. (The rightmost flip-flop is ‘to the 
left”’ of the leftmost flip-flop, forming a ring.) 


Chapter 15 


In the following instructions, roman numerals are states; the scanned symbol, if 
relevant, is in italics; the replacement symbol, if any, is in boldface type; R means 
move right, L means move left. 


15-1 


15-2 


15-4 


Add: (1 Y R38), (2 Y R38), (3 STOP). 
Add: (3 11.6), (61 L6), (670R7), (7 R8), (8 STOP). 


All digits are in pairs; the pair 11 symbolizes e, the pair 00 symbolizes binary 0; 
the pair 01 symbolizes binary 1. The instructions are: 

(LOR 2), (17 R83), (2R1), (8B R4), (40R 5), (4716), 

(65R4), GOLRY), (61017), (7016), (71 R8), (81 R9), 

(90 R10), (97 R11), (ORY), (11 R12), (12 STOP), 





















































Abacus, 6 

Abelson, R. P., 233, 234 
Absolute address, 59 

Access time of storage, 25 
Accounting machines, 22 
Accumulator, description of, 27 


‘Accuracy, of analog computer, 9 


of digital approximations, 7 
of digital computations, 24, 146 
AcsI-MATIC, 273 
ADD instruction, 32, 302 
Addition, of binary numbers, 108 
carries in, 7, 108 
logical circuit for, 282 
Address, absolute, 59 
definition of, 25 
Address code, definition of, 115 
Address modification, 39 
Advice taker, 98, 258 
Aho, E. J., 191 
Aiken, H., 5 
Air defense system, 239 
Air-traffic control, 256 
Algebraic compiler, 76-78 
in statistical calculation, 144, 148 
Algol, 75 
Algorithmic compiler, 76 
Alphanumeric characters, 115, 130 
example of unpacking of, 126 
on punched cards, 21 
Analog computer, 8 


index 


Analog data, 159 
Analog-to-digital converter, 10, 24, 159 
Analysis of variance, 144 
Analytic engine, 4 
And-gates, 277 
Apparent movement, 241 
Approximation, discrete, 7 
polynomial, 148 
series, 149 
Apt, 76 
Arc, 159 
Arithmetic, binary, 108 
complement, 111 
Arithmetic instruction, 32-38, 302 
Arithmetic statement in Fortran, 79 
Arithmetic unit, 27 
Artificial intelligence, 89, 197 
Assembly-line balancing, 98, 227 
Assembly listing, definition of, 62 
Assembly program, 61, 137, 148 
Attitude change, 233 
Auditory recognition, 189 
Auditory stimuli, 179 
Autocorrelation, 159 
Automation, 238, 253 
for efficiency, 244 
Automaton, 197 
Available-space list, 91 


B rogiater, 27, 127 
Babbage, C,, 4 
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Babcock, M. L., 199 
Backus, J., 75 
Baker, C. A., 251 
Bales, R. F., 235 
Bartee, T. C., 277, 285 
Base of numbers, 109 
Base address, 46 
Baseball, 98, 258-274 
Bell Telephone Laboratories, 5 
Bernstein, A., 229 
Binary arithmetic, 108 
Binary-choice experiment, 232 
Binary code, 22, 103-117 
Binary computer, definition of, 8, 106 
Binary counting, 108, 279, 289 
Binary-decimal translation, 107 
Binary digit, 7, 103 
Binary number system, 7, 107-109 
Binary point, 108 
Birdsall, T. G., 207, 209 
Bit, 7, 103 

(See also Information theory) 
Block diagram, 63 
Boneau, C. A., 158 
Bookkeeping, example of, 52 
Bootstrapping, definition of, 133 
Boredom in man-computer systems, 

238 

Boring, E. G., 197 
Bostrom, C., 161 
Bottenbruch, H., 76 
Box, G. E. P., 171 
British Museum algorithm, 221 
Brown, G. W., 164 
Bruner, J. S., 211 
Burks, A. W., 199 
Bush, R. R., 156 


Calculation errors, 9 
Calling sequence, definition of, 51 
Calliscope, 191 
Camera, 181 

Beattie, 192, 193 

Fairchild, 193 

for graphs, 145 

Triad, 193 
Card, punched, 5, 20, 21 
Card image, 130 
Carr, J. W., III, 97 
Carries in addition, 7, 108 
Catch trials, 208 


Cathode-ray tube (CRT), 24 
on Arc, 160 
displays for man-computer systems, 
251 
for graphs, 145 
technical details of, 190 
for visual output, 181 
Cattell, R. B., 155 
Cell, 25 
Cell assemblies, 199 
Central-limit theorem, 171 
Chapanis, A., 251 
Chapman, R. L., 256 
Charactron, 190 
Check out, 73 
(See also Debugging) 
Check sums, 133 
Checkers, 228 
Chess, 228 
computability of, 298 
Chomsky, Carol, 259 
Circuit elements, 277 
Clark, W. A., 159, 202, 209 
Clinical diagnosis, 161-162 
Clip, 76 
Clutter, definition of, 241 
Cosou, 76 
versus natural language, 272 
Code, binary, 22, 103-117 
Coleman, J. 8., 234 
Comrrt, 97 
Command (instruction), 30-34 
Common sense, 244 
Communalities, iteration for, in factor 
analysis, 150 
Communication, with computer, 23, 77, 
258-274 
in Sacg, 244 
voice, 248, 252 
Communications of the Association for 
Computing Machinery, 65 
Compilers, 15, 76-78 
algebraic, 76-78 
in statistical calculations, 144, 148 
Complement arithmetic, 111 
COMPLEMENT instruction, 121, 304 
Computability, 287-300 
and efficiency, 298 
storage requirements in, 11, 205 
Computer console, 134 
Computers, special-purpose, 12, 150, 200 
Concept formation, 283 
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